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the  Twitter  network  from  Yemen  over  the  period  from  October  2013  to  January  2014,  to 
assess  both  support  for  extremist  groups  and  support  to  the  Yemeni  government.  From 
our  research,  we  conclude  that  social  media  data,  when  combined  with  polling,  has  a 
positive  impact  on  analysis.  It  can  also  be  a  reliable  source  of  stand-alone  data  for 
evaluating  popular  support  under  certain  conditions.  Therefore,  we  recommend  future 
research  projects  focus  on  improving  the  quality  of  social  media  data  and  on  operational 
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I. 


INTRODUCTION 


Since  nations  have  existed,  they  have  faced  population-centric  fonns  of  warfare — 
rebellion,  insurgency,  and  civil  war — in  which  insurgents  typically  battle  entrenched 
governments  for  the  loyalty  of  the  populace,  with  each  side’s  movements  and  intentions 
obscured  by  the  fog  of  conflict.  In  the  modern  era,  governments  rely  on  laborious,  time- 
consuming,  and  expensive  opinion  polls  to  navigate  through  this  fog.  Opinion  polls  are 
like  green  and  red  marker  buoys  in  a  channel,  spaced  intermittently  along  the  way  to  help 
captains  determine  their  positions.  Like  buoys,  polling  is  reliable  and  sheds  light  on 
known  positions  in  time,  but  polls  are  inherently  spaced  apart  from  one  another  and  thus 
lack  timeliness.  In  a  sea  covered  by  a  thick  blanket  of  fog,  buoys  can  easily  be  obscured. 
Like  captains  navigating  in  unclear  conditions,  military  commanders  navigating  through 
the  fog  of  conflict  require  timely  location  information,  or  else  their  efforts  may  founder. 

Our  research  attempts  to  detennine  whether  social  media  data  analysis  can  serve 
as  a  proxy  for  or  fill  in  the  gap  between  opinion  polls.  If  so,  military  coimnanders  could 
continuously  monitor  population  sentiment  and  determine  whether  their  efforts  were 
affecting  the  population. 

To  answer  our  questions,  we  developed  a  method  that  reliably  measures  the 
geographic  distribution  of  aggregate  popular  sentiments  using  data  derived  from  public 
messages  sent  through  the  Twitter  network.  We  then  compared  these  measurements  to 
the  spatial  distribution  of  sentiments  derived  from  existing  public  opinion  polls  collected 
during  the  same  period. 

For  robustness,  we  tested  our  approach  in  Yemen,  a  difficult  test  country. 
Compared  to  Western  countries,  Yemen  has  low  levels  of  Internet  access  and  social 
media  users.  The  country  is  also  beset  by  low  levels  of  governance  and  violent  extremist 
organizations  battling  for  the  loyalty  of  the  populace.  Yemen  is  typical  of  a  country  that 
the  U.S.  military  would  be  tasked  to  deal  with  because  it  is  a  breeding  ground  for 
terrorism. 
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In  total,  the  data  used  to  develop  our  new  metrics  consisted  of  over  1.4  million 
tweets,  observed  in  Yemen  over  the  period  October  2013  to  January  2014.  From  our 
research  we  conclude  that  social  media  data,  when  combined  with  traditional  polling 
approaches,  can  greatly  improve  our  situational  awareness  over  what  can  be  learned  from 
polling  alone.  Under  some  circumstances,  it  may  be  possible  to  use  social  media  data  as  a 
stand-alone  tool  for  evaluating  negative  sentiment  for  a  standing  government  or  violent 
extremist  organizations  vying  for  control  of  a  country.  However,  we  also  find  that  our 
method  performed  better  at  identifying  geographic  patterns  of  negative  feelings  versus 
positive  feelings  regarding  the  government  of  Yemen  and  violent  extremist  organizations. 

We  recommend  further  research  focus  on  improving  the  algorithm  that  we 
developed.  Specifically,  we  recommend  testing  different  timeframes  in  Yemen,  different 
countries,  and  Western  countries.  Finally,  we  recommend  military  commanders  make 
greater  efforts  to  combine  the  use  of  social  media  data  and  more  traditional  polling 
approaches.  Doing  so  will  allow  commanders  to  better  focus  counterinsurgency 
operations  on  the  disaffected  portions  of  the  population  where  partnerships  are  most 
critical. 
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II.  LITERATURE  REVIEW 


A.  MILITARY  DOCTRINE  AND  ASSESSMENT 

This  project  was  inspired  in  large  part  by  the  team’s  combined  50  years  of  service 
in  both  the  U.S.  Navy  and  U.S.  Army,  working  against  asymmetric  actors.  Our 
experience  waging  counterinsurgency  operations  across  the  globe  has  taught  us  that 
sentiment  is  fundamental  in  gauging  success  in  the  type  of  warfare  the  U.S.  has  been 
focused  on  for  decades.1  Regrettably,  we  have  watched  numerous  Special  Forces 
commanders  struggle  to  evaluate  the  strategic  effectiveness  of  their  counterinsurgency 
operations.  We  feel  this  is  mainly  because  they  did  not  have  enough  information  on  the 
populace  in  the  regions  where  they  held  responsibility.  Compounding  this  issue  is  the  fact 
that  U.S.  Joint  Doctrine  Publications  instruct  commanders  to  measure  the  effectiveness  of 
their  operations  as  part  of  a  feedback  loop,  but  fail  to  provide  clear  ideas  of  how  to  do  the 
job  or  resources  to  help.2 

A  portion  of  the  doctrine  issue  we  illuminate  may  be  due  to  the  disagreement 
regarding  whether  to  measure  sentiment  in  the  first  place.  Some  would  argue  there  is  no 
difference  between  conventional  military  Clausewitzian  force-on-force  conflicts  and  a 
“people’s  war”  (e.g.,  low-intensity  conflict,  hybrid  warfare,  irregular  warfare,  etc.).34 
Therefore,  the  military  should  focus  simply  on  the  enemy,  making  popular  support 
irrelevant.5  Other  scholars  and  theorists,  such  as  counterinsurgency  warfare  experts, 
would  argue  that  only  the  actor  who  wins  and  maintains  popular  support  will  achieve  his 

1  Seth  G.  Jones,  RAND  Counterinsurgency  Study,  vol.  4,  Counterinsurgency  in  Afghanistan  (Santa 
Monica,  CA:  RAND  Corporation,  2008),  7. 

2  Jack  D.  Kem,  “Assessment:  Measures  of  Performance  and  Measures  of  Effectiveness,”  Military 
Intelligence  Professional  Bulletin  35,  no.  2  (2009):  48-50. 

3  Stuart  Kinross,  “Clausewitz  and  Low-Intensity  Conflict,”  Journal  of  Strategic  Studies  27,  no.  1 
(2004):  35-58;  Frank  Hoffman,  “Hybrid  Warfare  and  Challenges,”  JFQ  52,  no.  1  (2009);  James  Clancy 
and  Chuck  Crossett,  “Measuring  Effectiveness  in  Irregular  Warfare,”  Parameters  37,  no.  2  (2007):  88. 

4  Clausewitz  also  spoke  about  the  “passions  of  the  people.”  The  passions  of  the  people  were  a  key  part 
of  his  trinity  of  war. 

5  Raphael  Cohen,  “Just  How  Important  Are  ‘Hearts  and  Minds’  Anyway?  Counterinsurgency  Goes  to 
the  Polls,”  Journal  of  Strategic  Studies  37,  no.  4  (June/July  2014):  609-636;  Russell  Weigley,  The 
American  Way  of  War:  A  History  of  United  States  Military  Strategy  and  Policy  (Bloomington,  IN:  Indiana 
University  Press,  1977). 
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or  her  objectives.  This  is  believed  because,  in  Roger  Trinquier’s  words,  “the  ability  to 
exercise  power  over  a  populace  depends  on  the  tacit  or  explicit  agreement  of  the 
population  and  in  worst-case  scenarios  on  their  submission.”6 

Moreover,  among  those  who  do  not  question  the  relevance  of  population-centric 
counterinsurgency  warfare,  there  is  disagreement  about  whether  to  study  quantifiable 
metrics  or  qualitative  “hearts  and  minds”  metrics,  similar  to  the  broader  divisions  found 
across  the  social  sciences.7 

Military  units  that  value  behavior-based  metrics,  such  as  Psychological 
Operations,  have  developed  an  approach  called  effects-based  operations,  which  focuses 
on  observable  behavior-based  metrics  to  assess  military  effectiveness.8  They  analyze 
variables  like  security,  economic  indicators,  justice  indicators,  and  governance 
indicators.9  To  do  so,  they  gather  and  assess  public  attitudes,  beliefs,  atmospherics,  and 
opinions  via  three  means:  mass-media  analysis,  open-source  analysis,  and  opinion 
polling.  Passive  observation  of  a  local  population  and  mass-media  open-source  analysis 
(one-to-many  communication  modes)  provide  qualitative  insight  regarding  the  attitudes, 
beliefs,  and  opinions  of  key  demographics  and  elites  respectively.10 


6  George  Casey  Jr.  et  at.  Assessing  War:  The  Challenge  of  Measuring  Success  and  Failure 
(Washington,  DC:  Georgetown  University  Press,  2015),  218. 

7  Robert  B.  Sotire,  Measuring  Performance  and  Effectiveness  in  Irregular  Warfare:  Preventing 
Dysfunctional  Behavior  (Newport,  RI:  Naval  War  College,  2009). 

8  Edward  Smith,  “Effects-Based  Operations,”  in  Applying  Network-Centric  Warfare  in  Peace,  Crisis 
and  War  (Washington,  DC:  DOD  CCRP,  2002). 

9  Jason  Campbell  and  Jeremy  Shapiro,  “Afghanistan  Index:  Tracking  Variables  of  Reconstruction  and 
Security  in  Post-9/1 1  Afghanistan,”  Brookings  Institution,  October  28,  2008,  4-49;  Ethan  B.  Kapstein, 
“Measuring  Progress  in  Modern  Warfare,”  Surx’ival  54,  no.  1  (03/01,  2012):  137-158;  M.  Treblicock  and 
R.  Daniels,  Rule  of  Law  Reform  and  Development:  Charting  the  Fragile  Path  of  Progress  (Surrey,  UK: 
Edward  Elgar  Publishing,  2008),  42;  Craig  Cohen,  “Measuring  Progress  in  Stabilization  and 
Reconstruction,”  United  States  Institute  of  Peace:  Stabilization  and  Reconstruction  Series  (2003):  1-12; 
Michael  Dziedzic,  Barbara  Sotirin,  and  John  Agoglia,  eds.,  “Measuring  Progress  in  Conflict  Environments 
(MPICE),”  United  States  Institute  of  Peace,  March  19,  2008. 

10  Michael  T.  Flynn,  Matthew  F.  Pottinger,  and  Paul  D.  Batchelor,  Fixing  Intel:  A  Blueprint  for 
Making  Intelligence  Relevant  in  Afghanistan  (Washington,  DC:  Center  for  a  New  American  Security, 
2010).  doi:ADA511613;  John  Urry,  Mobilities  (Cambridge,  UK:  Polity,  2007);  Robert  Steele,  “Open 
Source  Intelligence,”  in  Handbook  of  Intelligence  Studies,  edited  by  Loch  K.  Johnson  (Oxford,  UK: 
Routledge,  2007),  129-147. 
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Proponents  of  effects-based  operations  assess  support  to  military  operations 
through  behavior-based  indicators.  This  approach  is  commonly  called  “hearts  and 
minds,”  because  it  measures  attitudes,  beliefs,  and  opinions.11  However,  critics  of  effects- 
based  operations  point  out  questionable  relationships  between  observable  behavior  and 
popular  support  indicators.12 

B.  SOCIAL  MEDIA  ANALYSIS 

An  alternative  approach  to  assessment,  not  considered  within  doctrines,  may  be 
provided  through  social  media  analysis.  Social  scientists  now  have  access  to  massive 
quantities  of  data  sourced  through  public  Internet  and  social  media  sites.  The  techniques 
used  to  comb  through  the  hard  drives  packed  with  data  have  various  titles,  which  can  be 
organized  under  the  broad  heading  of  data  mining.13  In  general  terms,  data  mining  is  a 
process  of  description  and  prediction  that  allows  analysts  to  sort,  analyze,  and  visualize 
data.14  One  area  of  knowledge  that  builds  from  data  mining  is  deriving  meaning  from 
written  text.  This  has  been  called  sentiment  analysis,  opinion  mining,  and  subjectivity 
analysis  15  We  refer  to  Georgios  Paltoglou,  a  senior  lecturer  from  the  University  of 
Wolverhampton,  United  Kingdom,  who  defines  it  as  follows: 

Specifically,  opinion  mining  addresses  the  problem  of  detecting, 
extracting,  analyzing,  and  quantifying  expressions  of  private  states  in 
written  text  in  an  automatic,  computer-mediated  fashion.  Particular 
emphasis  should  be  placed  on  the  term  “computer-mediated,”  as  the  field 


1 1  David  Galula,  Counterinsurgency  Warfare:  Theory  and  Practice  (Greenwood  Publishing  Group, 
2006);  United  States  Joint  Chiefs  of  Staff,  Joint  Publication  Counterinsurgency  (JP  3-24)  (Washington, 
DC:  U.S.  Government  Printing  Office,  2013);  Jordan  Stern,  Civil  Military  Operations  and  Military 
Information  Support  Operations  Coordination:  A  Non-Kinetic  Ballast  for  Disciplined  Counterinsurgency 
Operations  (Bethseda,  MD:  Small  Wars  Foundation,  2011). 

12  Christopher  Paul  and  William  Marcellino,  “Dominating  Duffer’s  Domain:  Lessons  for  the  21st- 
Century  Information  Operations  Practitioner,”  RAND  Corporation,  Santa  Monica:  CA,  2016. 

1 3  Usama  Fayyad,  Gregory  Piatetsky-Shapiro,  and  Padhraic  Smyth,  “From  Data  Mining  to  Knowledge 
Discovery  in  Databases.”  AI Magazine  17,  no.  3  (1996):  37. 

14  Alexander  Furnas,  “Everything  You  Wanted  to  Know  about  Data  Mining  but  Were  Afraid  to  Ask,” 
Atlantic,  April  3,  2012. 

^  Bo  Pang  and  Lillian  Lee.  “Opinion  Mining  and  Sentiment  Analysis,”  Foundations  and  Trends  in 
Information  Retrieval  2,  no.  1-2  (2008):  1—135. 
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has  a  particular  focus  on  designing,  analyzing,  and  implementing  software 
that  perfonns  the  aforementioned  analysis  in  an  automatic  manner.16 

There  are  many  social  media  sources  for  data  mining,  including  social  networking 
(e.g.,  Facebook),  bookmarking  (e.g.,  Pinterest),  social  news  (e.g.,  Reddit),  media  sharing 
(e.g.,  YouTube),  microblogging  (e.g.,  Twitter),  and  various  blogs/forums.17  Researchers 
from  numerous  fields  are  looking  deeper  into  questions  using  big  data  from  social  media 
to  study  behavior  in  humans,  politics,  financial  markets,  and  public  opinion,  and 
movement  patterns  of  groups  to  identify  the  spread  of  diseases,  migration  patterns  of 
animals,  patterns  of  behavior  in  humans,  and  cross-border  routes  (e.g.,  smuggling 


16  Georgios  Paltoglou,  “Sentiment  Analysis  in  Social  Media,”  in  Online  Collective  Action,  Dynamics 
of  the  Crowd  in  Social  Media,  eds.  N.  Agarwal,  M.  Lim,  and  R.  T.  Wigard  (Vienna,  Austria:  Springer 
International  Publishing,  2013),  3-17. 

17  Alexander  Pak  and  Patrick  Paroubek.  “Twitter  as  a  Corpus  for  Sentiment  Analysis  and  Opinion 
Mining,”  in  Proceedings  of  the  Seventh  Conference  on  International  Language  Resources  and  Evaluation 
(LREC  ‘10),  eds.  Nicoletta  Calzolari  et  al  (Valletta,  Malta:  European  Language  Resources  Distribution 
Agency,  May  2010);  Fayyad,  Piatetsky-Shapiro,  and  Smyth,  “From  Data  Mining  to  Knowledge 
Discovery”;  Stefan  Stieglitz  and  Linh  Dang-Xuan,  “Political  Communication  and  Influence  through 
Microblogging:  An  Empirical  Analysis  of  Sentiment  in  Twitter  Messages  and  Retweet  Behavior,”  in 
Proceedings  of  the  45th  Hawaii  International  Conference  on  System  Science,  ed.  Ralph  H.  Sprague  Jr. 
(Washington,  DC:  IEEE  Computer  Society,  2012),  3500-3509;  Mike  Thelwall,  Kevan  Buckley,  and 
Georgios  Paltoglou,  “Sentiment  in  Twitter  Events,”  Journal  of  the  American  Society  for  Information 
Science  and  Technology  62,  no.  2  (2011):  406-418;  Xiaofeng  Wang,  Matthew  S.  Gerber,  and  Donald  E. 
Brown,  “Automatic  Crime  Prediction  Using  Events  Extracted  from  Twitter  Posts,”  in  SBP'12  Proceedings 
of  the  5th  International  Conference  on  Social  Computing,  Behavioral-Cultural  Modeling  and  Prediction, 
eds.  Shanchieh  Jay  Yang,  Ariel  M.  Greenberg,  and  Mica  Endsley  (Berlin,  Germany:  Springer- Verlag 
Berlin,  Heidelberg,  2012),  231-238. 
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routes).18  Other  areas  of  research  include  the  relationship  between  social  media  and 
individual  or  group  behavior,  which  can  potentially  assess  the  impact  of  online  activism 
or  a  government’s  public  relations  campaign.19  There  are  also  efforts  to  understand  social 
mobilization  and  collective  violence,  of  particular  value  in  assessing  a  population  through 
social  movement  theory.20  However,  many  of  these  studies  have  faced  substantial 
barriers.  For  instance,  most  studies  focus  on  Western  countries  that  use  the  English 
language  and  are  dominated  by  Western  culture.21 


18  Morgan  R.  Frank  et  al.,  “Happiness  and  the  Patterns  of  Life:  A  Study  of  Geolocated  Tweets,” 
Scientific  Reports  3  (2013);  Bartosz  Hawelka  et  al.,  “Geo-Located  Twitter  as  Proxy  for  Global  Mobility 
Patterns,”  Cartography  and  Geographic  Information  Science  41,  no.  3  (2014):  260-271;  Marcel  Salathe  et 
al.,  “Influenza  A  (H7N9)  and  the  Importance  of  Digital  Epidemiology,”  New  England  Journal  of  Medicine 
369,  no.  5  (Aug  1,  2013):  401-4;  Justine  I.  Blanford  et  al.,  “Geo-Located  Tweets:  Enhancing  Mobility 
Maps  and  Capturing  Cross-Border  Movement,”  PloS  ONE  10,  no.  6  (Jun  2015,  2015);  Zhiyuan  Cheng, 
James  Caverlee,  and  Kyumin  Lee,  “You  Are  Where  You  Tweet:  A  Content-Based  Approach  to  Geo- 
Locating  Twitter  Users,”  in  Proceedings  of  the  19th  ACM  International  Conference  on  Information  and 
Knowledge  Management,  2010,  759-768;  Ryan  Compton,  David  Jurgens,  and  David  Allen,  “Geotagging 
One  Hundred  Million  Twitter  Accounts  with  Total  Variation  Minimization,”  Big  Data  (Big  Data),  2014 
IEEE  International  Conference  on,  2014,  393-401;  Michael  D.  Conover  et  al.,  “The  Geospatial 
Characteristics  of  a  Social  Movement  Communication  Network,”  PloS  ONE  8,  no.  3  (2013):  e55957; 
Andreas  Kaltenbrunner  et  al.,  “Far  from  the  Eyes,  Close  on  the  Web:  Impact  of  Geographic  Distance  on 
Online  Social  Interactions,”  Proceedings  of  the  2012  ACM  Workshop  on  Online  Social  Networks  (New 
York:  ACM,  2012),  19-24;  Juhi  Kulshrestha  et  al.,  “Geographic  Dissection  of  the  Twitter  Network,” 
ICWSM,  2012;  Ryong  Lee  and  Kazutoshi  Sumiya,  “Measuring  Geographical  Regularities  of  Crowd 
Behaviors  for  Twitter-Based  Geo-Social  Event  Detection,”  in  Proceedings  of  the  2nd  ACM  SIGSPATIAL 
International  Workshop  on  Location-Based  Social  Networks  (New  York:  ACM,  2010),  1—10;  Kalev 
Leetaru  et  al.,  “Mapping  the  Global  Twitter  Heartbeat:  The  Geography  of  Twitter,”  First  Monday  18,  no.  5 
(2013);  Lewis  Mitchell  et  al.,  “The  Geography  of  Happiness:  Connecting  Twitter  Sentiment  and 
Expression,  Demographics,  and  Objective  Characteristics  of  Place,”  PloS  ONE  8,  no.  5  (2013):  e64417; 
Stephen  C.  Nemeth,  Jacob  A.  Mauslein,  and  Craig  Stapley,  “The  Primacy  of  the  Local:  Identifying 
Terrorist  Hot  Spots  Using  Geographic  Information  Systems,”  Journal  of  Politics  76,  no.  2  (2014):  304- 
317;  Yuri  Takhteyev,  Anatoliy  Gruzd,  and  Barry  Wellman,  “Geography  of  Twitter  Networks,”  Social 
Networks  34,  no.  1  (2012):  73-81;  Quan  Yuan  et  al.,  “Who,  Where,  When  and  What:  Discover  Spatio- 
Temporal  Topics  for  Twitter  Users,”  in  Proceedings  of  the  19th  ACM  SIGKDD  International  Conference 
on  Knowledge  Discovery  and  Data  Mining,  eds.  Ingerjit  S.  Dhillon  et  al.  (New  York:  ACM,  2013),  605- 
613. 

19  Raz  Schwartz  and  Germaine  R.  Halegoua,  “The  Spatial  Self:  Location-Based  Identity  Performance 
on  Social  Media,”  New  Media  &  Society,  April  9,  2014,  1-18.  doi:  10. 1 1 77/146 14448 1 453 1 364. 

20  Doowan  Lee,  "A  Social  Movement  Approach  to  Unconventional  Warfare,"  Special  Warfare  July- 
September,  Volume  26,  no.  Issue  3  (2013),  27-32.;  Doug  McAdam,  Political  Process  and  the  Development 
of  Black  Insurgency,  1930-1970,  ed.  2ndUniversity  of  Chicago  Press,  LTD.,  London,  1982). 

21  T.  Camber  Warren,  “Mapping  the  Rhetoric  of  Violence:  Political  Conflict  Discourse  and  the 
Emergence  of  Identity  Radicalization  in  Nigerian  Social  Media,”  prepared  for  presentation  at  the  Annual 
Meeting  of  the  American  Political  Science  Association,  Sept.  3-6,  2015,  San  Francisco,  CA. 
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Nevertheless,  large  strides  have  been  made  in  analyzing  individual  messages  to 
determine  underlying  sentiment,  which  is  the  focus  of  our  research.22  These  strides  in 
sentiment  studies  have  resulted  in  new  approaches,  including  natural  language 

processing,  text  analysis,  linguistic  inquiry,  word  count,  keyword  spotting,  lexical 

23 

affinity,  statistical  methods,  and  concept-level  techniques.  Keyword  spotting  is  the 

most  simplistic  approach,  because  it  sorts  text  by  category  using  unambiguous  emotional 

24 

words.  For  example,  if  keyword  software  is  programmed  to  look  for  the  word 
government,  this  is  because  the  word  government  has  already  been  associated  with  an 
emotional  category  like  good  or  bad.  Another  approach  is  lexical  affinity,  which  sorts 
text  much  the  same  way  as  keyword  spotting.  The  big  difference  is  that  it  is  able  to 


22  Ahmed  Abbasi,  Hsinchun  Chen,  and  Arab  Salem,  “Sentiment  Analysis  in  Multiple  Languages: 
Feature  Selection  for  Opinion  Classification  in  Web  Forums,”  ACM  Transactions  on  Information  Systems 
(TOIS)  26,  no.  3  (2008):  12;  Apoorv  Agarwal  et  ah,  “Sentiment  Analysis  of  Twitter  Data,”  in  Proceedings 
of  the  Workshop  on  Languages  in  Social  Media,  eds.  Meena  Nagarajan  and  Michael  Gamon,  Association 
for  Computational  Linguistics,  201 1,  30-38;  Younggue  Bae  and  Flongchiil  Lee,  “Sentiment  Analysis  of 
Twitter  Audiences:  Measuring  the  Positive  or  Negative  Influence  of  Popular  Twitterers,”  Journal  of  the 
American  Society  for  Information  Science  and  Technology  63,  no.  12  (2012):  2521-2535;  Albert  Bifet  and 
Eibe  Frank,  “Sentiment  Knowledge  Discovery  in  Twitter  Streaming  Data,”  in  DS’10  Proceedings  of  the 
13th  International  Conference  on  Discovery  Science,  eds.  Berharnd  Pfahringer,  Geoff  Holmes,  and  Achim 
Floffmann  (Fleidelberg,  Germany:  Springer- Verlag,  2010),  1-15;  Johan  Bollen,  Alberto  Pepe,  and  Huina 
Mao,  “Modeling  Public  Mood  and  Emotion:  Twitter  Sentiment  and  Socio-Economic  Phenomena,”  in 
Proceedings  of  the  Fifth  International  AAAI  Conference  on  Weblogs  and  Social  Media  (ICWSM  2011 ),  eds. 
N.  Nicolov  and  J.  G.  Shanahan  (Barcelona,  Spain:  AAAI  Press,  July  2011),  1-10;  Peter  Sheridan  Dodds  et 
ah,  “Temporal  Patterns  of  Flappiness  and  Information  in  a  Global  Social  Network:  Hedonometrics  and 
Twitter,”  PloS  ONE  6,  no.  12  (2011):  e26752;  Rui  Fan  et  ah,  “Anger  Is  More  Influential  than  Joy: 
Sentiment  Correlation  in  Weibo.”  PloS  ONE  9,  no.  10  (2014):  el  10184;  M.  Ghiassi,  J.  Skinner,  and  D. 
Zimbra,  “Twitter  Brand  Sentiment  Analysis:  A  Hybrid  System  Using  N-Gram  Analysis  and  Dynamic 
Artificial  Neural  Network,”  Expert  Systems  with  Applications  40,  no.  16  (2013):  6266-6282;  Scott  A 
Golder  and  Michael  W.  Macy.  “Diurnal  and  Seasonal  Mood  Vary  with  Work,  Sleep,  and  Daylength  across 
Diverse  Cultures,”  Science  333,  no.  6051  (2011):  1878-1881;  Shu  Huang  et  ah,  “Sentiment  and  Topic 
Analysis  on  Social  Media:  A  Multi-Task  Multi-Label  Classification  Approach,”  in  Proceedings  of  the  5th 
Annual  ACM  Web  Science  Conference  (New  York:  ACM,  2013),  172-181;  Long  Jiang  et  ah,  “Target- 
Dependent  Twitter  Sentiment  Classification,”  in  Proceedings  of  the  49th  Annual  Meeting  of  the  Association 
for  Computational  Linguistics:  Human  Language  Technologies,  vol.  1  (Stroudsburg,  PA:  Association  for 
Computational  Linguistics,  2011),  151-160;  Lewis  Mitchell  et  ah,  “The  Geography  of  Happiness”; 

Brendan  O’Connor  et  ah,  “From  Tweets  to  Polls:  Linking  Text  Sentiment  to  Public  Opinion  Time  Series,” 
in  Proceedings  of  the  Fourth  International  AAAI  Conference  on  Weblogs  and  Social  Media,  2010,  1-2. 

22  Erik  Cambria  et  ah,  “New  Avenues  in  Opinion  Mining  and  Sentiment  Analysis,”  IEEE  Intelligent 
Systems  no.  2  (2013):  15-21;  Bo  Pang,  Lillian  Lee,  and  Shivakumar  Vaithyanathan,  “Thumbs  Up? 
Sentiment  Classification  Using  Machine  Learning  Techniques,”  in  Proceedings  of  the  2002  Conference  on 
Empirical  Methods  in  Natural  Language  Processing  (Stroudsburg,  PA:  Association  for  Computational 
Linguistics,  2002);  Namrata  Godbole,  Manja  Srinivasaiah,  and  Steven  Skiena,  “Large-Scale  Sentiment 
Analysis  for  News  and  Blogs,”  Icwsm  7,  no.  21  (2007):  219-222. 

Andrew  Ortony,  Gerald  L.  Clore,  and  Allan  Collins,  The  Cognitive  Structure  of  Emotions  (New 
York:  Cambridge  University  Press,  1990). 
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measure  the  distance  between  words  in  a  text  string  and  assign  an  intensity  value  based 
on  the  distance.-' 

The  statistical  method  is  slightly  more  complex.  It  uses  natural  language 
processing  (NLP)  to  cull  through  massive  amounts  of  data  and  assigns  values  to  patterns 
in  the  text.  One  example  is  the  “bag  of  words”  approach,  which  analyzes  a  text  without 
considering  the  order  or  distance  between  words.  This  approach  essentially  counts  the 
frequency  of  words  co-occurrences,  and  then  infers  “negative”  or  “positive”  valence 
categories  based  upon  the  patterns  of  these  co-occurrences.  It  is  commonly  used  to  filter 
spam  by  using  a  list  of  valued  words  to  determine  whether  an  email  is  an  advertisement 

(i.e.,  negative  words)  or  a  legitimate  communication  (i.e.,  positive  words).-6 

The  most  complex  approaches  are  known  as  concept-level  techniques.  These 

techniques  primarily  focus  on  semantic  analysis  of  the  text.  This  means  that  they  evaluate 

27 

how  concepts  in  one  text  relate  to  concepts  in  another  text.  This  form  of  social  media 
analysis,  rooted  in  lexical  analysis,  examines  the  content  of  messages.28  Lexical 
analysis’s  subfields  and  analytical  methods  include  discourse  analysis,  narrative  analysis, 
trend  analysis,  and  sentiment  analysis.  Discourse  analysis  reviews  both  two-way 
conversations  and  one-way  rhetoric.  These  inputs  typically  flow  from  an  organization’s 
spokesperson  or  a  key  communicator  to  his  or  her  listeners,  readers,  or  viewers.29 
Narrative  analysis  highlights  how  an  individual  or  group  reflects  their  understanding  of 
their  place  in  the  world;  this  approach  is  increasingly  applied  within  the  field  of  social 


Hugo  Liu,  Henry  Lieberman,  and  Ted  Selker,  “A  Model  of  Textual  Affect  Sensing  Using  Real- 
World  Knowledge,”  in  Proceedings  of  the  2003  International  Conference  on  Intelligent  User  Interfaces 
(New  York:  ACM,  2003). 

26  Peter  Turney,  “Thumbs  Up  or  Thumbs  Down?  Semantic  Orientation  Applied  to  Unsupervised 
Classification  of  Reviews,”  in  Proceedings  of  the  40th  Annual  Meeting  of  the  Association  for 
Computational  Linguistics  (Stroudsburg,  PA:  Association  for  Computational  Linguistics,  2002),  417-424. 

22  Erik  Cambria  et  al.,  “Sentic  Patterns:  Dependency-Based  Rules  for  Concept-Level  Sentiment 
Analysis,”  Knowledge-Based  Systems  69,  (2014):  45-63. 

28  Pang  and  Lee,  “Opinion  Mining  and  Sentiment  Analysis.” 

29  Teun  A.  van  Dijk,  “Critical  Discourse  Analysis,”  in  The  Handbook  of  Discourse  Analysis,  eds. 
Deborah  Schiffrin,  Deborah  Tannen,  and  Heidi  E.  Hamilton  (Malden,  Massachusetts:  Blackwell  Publishers, 
2001,  349-371. 


9 


movement  theory.30  Trend  analysis  instead  focuses  on  temporal  patterns  in  discourse, 
mathematically  identifies  items  experiencing  accelerating  increases  in  frequency,  whether 
those  items  are  word  and  phrases  used  in  search  engines,  or  ‘trending’  stories  on  news 
sites.31 


The  output  of  sentiment  analysis  can  take  multiple  forms:  including  ordered, 
continuous,  and  categorical  values.  Ordered  outputs  might  be  either  positive  or  negative, 
or  might  include  a  ternary  division  between  positive,  negative,  and  neutral.32  Categorical 
values  are  similar,  but  they  assign  no  actual  numerical  content,  so  the  categories  are  just 
boxes  like  “a”  or  “cat,”  which  do  not  imply  that  one  value  is  numerically  greater  than 
another. 

Researchers  have  used  numerous  methods  to  analyze  and  visually  represent  social 
media  data.  Geospatial  analysis,  for  instance,  is  a  visual  representation  of  data  that  looks 
at  identifying  spatially  related  patterns  between  events  and/or  individuals  and  their 
locations.33  A  great  example  of  this  approach  is  heat  maps.  A  heat  map  is  a  graphical 
representation  of  data  in  which  the  individual  values  in  a  matrix  are  represented  as  colors. 
The  color  coding  highlights  distinctions  between  values  taken  by  variables  in  a  hierarchy. 


30  Catherine  K.  Riessman,  “Narrative  Analysis,”  in  M.S.  Lewis-Beck,  A.  Bryman,  and  T.  Futing  Liao, 
eds.,  SAGE  Encyclopedia  of  Social  Science  Research  Methods,  vol.  3  (Thousand  Oaks,  CA:  SAGE 
Publications,  2003);  Arthur  A.  Berger,  Medici  Analysis  Techniques  (Thousand  Oaks,  CA:  SAGE 
Publications,  2013);  Ruth  E.  Page,  Stories  and  Social  Media:  Identities  and  Interaction  (Oxford,  UK: 
Routledge,  2013). 

3 1  Jey  H.  Lau,  Nigel  Collier,  and  Timothy  Baldwin.  “On-Line  Trend  Analysis  with  Topic  Models: 
#Twitter  Trends  Detection  Topic  Model  Online,”  conference  paper,  Proceedings  of  COLING  2012, 
December  2012;  M.  Osborne  et  al.,  “BieberNo  More:  First  Story  Detection  Using  Twitter  and  Wikipedia,” 
in  SIGIR  2012  Workshop  on  Time-Aware  Information  Access  Oregon,  2012;  S.  Petrovic,  M.  Osborne,  and 
V.  Lavrenko,  “Streaming  First  Story  Detection  with  Application  to  Twitter,”  in  Proceedings  of  Human 
Language  Technologies:  The  1 1th  Annual  Conference  of  the  North  American  Chapter  of  the  Association 
for  Computational  Linguistics  (NAACL  HLT  2010)  (Stroudsburg,  PA:  Association  for  Computational 
Linguistics:  2010),  181-189. 

32  Anna  Chmiel  et  al.,  “Negative  Emotions  Boost  User  Activity  at  BBC  Forum,”  Physica  A:  Statistical 
Mechanics  and  Its  Applications  390,  no.  16  (2011):  2936-2944;  Marija  Mitrovic,  Georgios  Paltoglou,  and 
Bosiljka  Tadic,  “Quantitative  Analysis  of  Bloggers’  Collective  Behavior  Powered  by  Emotions,”  Journal 
of  Statistical  Mechanics:  Theory  and  Experiment  2011,  no.  02  (2011):  P02005. 

33  Anthony  Stefanidis,  Andrew  Crooks,  and  Jacek  Radzikowski,  “Flarvesting  Ambient  Geospatial 
Information  from  Social  Media  Feeds,”  Geojournal  78,  no.  2  (2013):  319-338;  Daniel  Sui  and  Michael 
Goodchild,  “The  Convergence  of  GIS  and  Social  Media:  Challenges  for  GIScience,”  International  Journal 
of  Geographical  Information  Science  25,  no.  11  (2011):  1737-1748;  Sarah  Elwood,  Michael  F.  Goodchild, 
and  Daniel  Z.  Sui,  “Researching  Volunteered  Geographic  Information:  Spatial  Data,  Geographic  Research, 
and  New  Social  Practice,”  Annals  of  the  Association  of  American  Geographers  102,  no.  3  (2012):  571-590. 


10 


Social  network  analysis  (SNA),  on  the  other  hand,  uses  social  media  data  to  identify  the 
ties  between  actors  and  the  strengths  of  those  ties.34  The  ties  are  frequently  displayed  in  a 
hub-and-spoke  wheel  of  actors  and  their  connections  to  other  actors  in  a  network. 

A  variety  of  issues  arise  when  attempting  to  determine  sentiment  using  the 
Internet  and  social  media,  including  the  source,  sample  representation,  and  language.  It  is 
important  to  note  that  data  derived  from  these  sources  often  do  not  represent  proper 
“samples”  in  the  statistical  sense.35  Source  issues  involve  not  knowing  where  the  data 
originates  from.  Imagine  a  pollster  asking  questions  of  a  person  who  is  standing  behind  a 
white  sheet.  The  pollster  cannot  know  for  certain  the  identity  of  the  source.  Data  found 
online  may  have  all  kinds  of  unwanted  sources  including  hots,  Internet  agents,  wanderers, 
brokers,  trolls,  propagandists,  and  even  official  news  outlets  that  can  skew  the  results.36 
Additionally,  in  demographic  terms,  “Internet  and  social  media  users  are  not  likely  a  true 
representation  of  a  given  population,”  and  as  a  result  they  do  not  necessarily  provide  a 
representative  sample  for  analysis.37  For  instance,  access  to  the  Internet  in  many  countries 
may  only  be  found  in  urban  areas  with  improved  infrastructure.  People  from  rural,  less 
connected  areas  may  therefore  be  underrepresented.  Other  demographic  issues  related  to 
Internet  and  social  media  access  includes  age,  education,  gender,  and  income.  Still,  there 
are  widely  accepted  statistical  techniques  for  dealing  with  such  issues.  As  we  explain 
more  fully,  it  may  be  possible  reduce  such  difficulties  by  using  ‘control  variables’  to  hold 
certain  factors  constant  when  examining  the  effects  of  a  variable  of  interest. 


34  Sean  F.  Everton,  Disrupting  Dark  Networks,  vol.  34  (New  York:  Cambridge  University  Press, 
2012);  Stuart  Koschade,  “A  Social  Network  Analysis  of  Jemaah  Islamiyah:  The  Applications  to 
Counterterrorism  and  Intelligence,”  Studies  in  Conflict  &  Terrorism  29,  no.  6  (2006):  559-575;  Stanley 
Wasserman  and  Katherine  Faust,  Social  Network  Analysis:  Methods  and  Applications,  vol.  8  (New  York: 
Cambridge  University  Press,  1994). 

35  Stephen  Ansolabehere  and  Eitan  Hersh,  “Validation:  What  Big  Data  Reveal  about  Survey 
Misreporting  and  the  Real  Electorate,”  Political  Analysis  (2012):  mps023;  Pablo  Barbera,  “How  Social 
Media  Reduces  Mass  Political  Polarization:  Evidence  from  Germany,  Spain,  and  the  US,”  working  paper 
prepared  for  the  2015  APSA  Conference,  New  York  University,  2014;  Alan  Mislove  et  al.,  “Understanding 
the  Demographics  of  Twitter  Users,”  ICWSM  1 1  (2011):  5. 

36  Fah-Chun  Cheong,  Internet  Agents:  Spiders,  Wanderers,  Brokers,  and  Bots  (San  Francisco,  CA: 
New  Riders  Publishing,  1996). 

37  Ansolabehere  and  Hersh,  “Validation:  What  Big  Data  Reveal”;  Pablo  Barbera  and  Gonzalo  Rivero, 
“Understanding  the  Political  Representativeness  of  Twitter  Users,”  Social  Science  Computer  Review  33, 
no.  6  (December  1,  2015):  712-729;  Mislove  et  al.,  “Understanding  the  Demographics  of  Twitter  Users.” 


11 


Since  most  social  networking  platforms  were  built  by  English  speakers,  the 
diversity  of  written  fonnats  seen  in  other  languages  can  pose  its  own  challenges. 
Processing  English  text  is  relatively  easy  compared  to  processing  text  in  some  foreign 
languages.  English  uses  a  relatively  simple  alphabet  that  contains  separate  letters  for 
consonants  and  vowels.  Other  writing  forms,  for  instance  Chinese  logograms,  are  built  of 
glyphs  that  represent  words.  A  glyph  is  a  meaningful  component  of  a  word,  rather  than  a 
phonetic  element.  Ideographic  writing  is  based  on  pictures  that  represent  concepts  or 
ideas,  rather  than  specific  words.  In  contrast,  a  segmental  script,  like  Arabic,  uses 
graphemes  that  represent  phonemes,  distinct  units  of  sound  that  distinguish  one  word 
from  another.38 

Given  these  difficulties,  one  approach,  taken  by  many  existing  works,  is  to  stick 
with  English-language  databases.  However,  when  faced  with  a  research  question  that 
requires  analysis  of  foreign  societies,  such  an  approach  seems  too  sharply  limiting. 
Promisingly,  recent  advances  in  machine  translation  have  increased  the  ease  of 
generating  relatively  reliable  translations  of  single  words  and  short  phrases.39  For 
instance,  researchers  can  access  automated  machine-generated  translations  through  online 
tools,  such  as  the  Google  Translate  API.  This  service  allows  for  simple  translation  from  a 
wide  variety  of  languages  to  other  languages.  More  nuanced  approaches  to  machine 
translation  utilize  natural  language  parsers  to  consider  the  broader  grammatical  structures 
of  sentences  and  paragraphs.  However,  this  requires  careful  tokenization  of  the  text 
combined  with  language-specific  grammar  rules,  which  can  make  this  approach  cost- 
prohibitive  in  computational  terms  when  face  with  very  large  textual  corpora. 


38  Michael  A.  Halliday,  Spoken  and  Written  Language  (Oxford,  UK:  Oxford  University  Press,  1989). 

39  Mohammad  Salameh,  Saif  M.  Mohammad,  and  Svetlana  Kiritchenko,  "Sentiment  after  Translation: 
A  Case-Study  on  Arabic  Social  Media  Posts,"  in  Proceedings  of  the  2015  Conference  of  the  North 
American  Chapter  of  the  Association  for  Computational  Linguistics:  Human  Language  Technologies 
(2015):  767-777. 
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III.  BACKGROUND— YEMEN 


Yemen  is  an  ideal  place  to  test  our  research.  Compared  to  Western  countries, 
Yemen  has  low  levels  of  Internet  access  and  few  social  media  users.  The  country  is  also 
struggling  with  low  levels  of  governance  and  violent  extremist  organizations  battling  for 
the  loyalty  of  the  populace.  Yemen  is  typical  of  a  country  that  the  U.S.  military  would  be 
tasked  to  deal  with,  because  it  is  a  breeding  ground  for  terrorism.  The  following  section 
delves  into  the  problems  facing  Y emen. 

For  the  majority  of  the  20th  century,  Yemen  was  divided  between  the  Ottoman 
and  British  empires.  Following  the  fall  of  the  Ottoman  Empire,  northern  Yemen  became 
the  Yemen  Arab  Republic.  Southern  Yemen  (People’s  Democratic  Republic  of  Yemen) 
remained  a  British  colony  until  1967.  Finally,  in  1990,  northern  and  southern  Yemen 
united.  Four  years  after  becoming  a  united  nation,  Yemen  experienced  a  two-month-long 
civil  war  over  governance. 

Since  the  1994  civil  war,  Yemen  has  become  increasingly  unstable.  Violent 
extremist  organizations  have  filled  gaps  where  the  government  has  failed.  According  to 
the  Fund  for  Peace,  the  country  was  ranked  in  the  top  10  most  fragile  in  both  2013  and 
2014.40  Numerous  organizations  are  vying  for  control  of  the  weak  Yemeni  government. 

A.  VIOLENT  EXTREMIST  ORGANIZATIONS 

Violent  extremist  organizations  (VEO)  in  Yemen  include  the  Shia  Houthis,  Sunni 
Salafi,  and  al-Qaeda  in  the  Arabian  Peninsula  (AQAP).  All  three  VEOs  have  challenged 
the  government  over  control  of  territory,  threatened  government  progress  initiatives  such 
as  the  National  Dialogue  Conference  (NDC),  conducted  kidnapped  for  ransom,  murdered 
civilians,  and  leveraged  poor  socioeconomic  conditions  and  government  corruption, 
leaving  Yemen  unstable  and  a  security  concern  for  the  rest  of  the  world  and  organizations 
that  want  to  help.  Non-governmental  agencies  are  loath  to  help  for  fear  of  their  safety  on 
the  ground.  For  example,  on  November  14,  2013,  VEOs  sabotaged  the  Internet  and  cell 

40  Fund  for  Peace,  “The  Fragile  States  Index  2015,”  accessed  June  15,  2016, 
http://fsi.fundforpeace.org/rankings-2014. 
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phone  towers  in  the  Shabwa  Governorate.41  The  Yemeni  population  suffered  as  a  result, 
and  the  government’s  response  was  popularly  viewed  as  insufficient.  The  government 
claimed  many  excuses  for  their  lack  of  action,  including  insufficient  resources,  security 
concerns,  and  roads  barricaded  by  the  VEOs.  Ultimately,  the  government  missed  the 
opportunity  to  exploit  VEO  violent  actions  via  propaganda  or  other  mean,  leaving  the 
VEOs  impervious  to  public  scrutiny,  despite  their  attacks  on  public  infrastructure. 

B.  HOUTHIS 

The  Shia  Houthis  believe  they  are  direct  descendants  of  the  Prophet  Muhammad, 
making  them  the  lawful  rulers  of  Yemen.42  Needless  to  say,  this  places  them  at  odds  with 
the  government.  Because  of  their  violent  actions,  they  are  deemed  insurgents. 
Compounding  the  discontent  stemming  from  the  Houthis’  religious  justification,  the 
government  has  intentionally  marginalized  the  group,  placing  them  at  odds  with  the  rest 
of  society  and  reducing  their  socioeconomic  significance.  Marginalization  of  the  Houthis 
began  with  Saleh,  president  of  Yemen  from  1990  to  2012,  and  is  believed  to  be  the 
foundation  for  the  2004  Yemen  civil  war  43 

Saudi  Arabia  has  generally  viewed  the  Houthis  as  a  threat  and  readily  agreed  to 
the  Yemeni  government’s  request  to  bomb  the  Houthis’  home  territory  along  the  Saudi 
border  in  November  2009.  Additionally,  Saudi  Arabia  is  claimed  to  have  contributed 
between  $200  and  $300  million  annually  in  security  assistance  to  President  Saleh.44 
Saudi  Arabia  was  also  involved  in  exerting  Wahhabi  influence  in  Yemen  and  most 
recently  in  the  northern  Sa’ada  Governorate  45  The  Houthis  remain  wary  of  attacks  from 
Saudi  Arabia  and  the  Yemeni  Army. 


41  Ibid. 

42  U.S.  Government,  U.S.  Military,  and  Department  of  Defense,  Yemen  in  Perspective  -  Orientation 
Guide  and  Yemeni  Cultural  Orientation:  Geography,  History,  Economy,  Security,  Customs,  Aden,  Sanaa, 
Sunni,  and  Shi’ a,  Sufism,  Qat  Chewing,  Al-Qaeda,  Houthi  Rebellion  (Cleveland,  OH:  Progressive 
Management,  2015),  1-658,  Kindle  edition. 

43  Ibid. 

44  Ibid. 

45  Ibid. 
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Iran,  on  the  other  hand,  generally  views  the  Houthis  as  useful  proxies  against  the 
Saudis.  Interestingly,  Iran  and  the  Houthis  do  not  follow  the  same  Shia  ideology.  Iran 
conveniently  ignores  the  fact  that  the  Houthis  follow  Fiver  Shia,  which  does  not 
acknowledge  later  imams,  while  the  Iranians  believe  in  Twelver  Shia.  Despite  this 
religious  difference,  Iran  is  suspected  of  providing  financial  funding,  military  training, 
and  material  aid  to  the  Houthis  via  Hezbollah.46 

C.  SALAFIS 

Sunni  Salafism  was  established  around  the  middle  of  the  19th  century  in  Egypt.47 
Salafi  Muslims  moved  to  Yemen  about  three  decades  ago.48  They  believe  in  re¬ 
establishing  ancient  traditions  of  Islam  and  do  not  believe  in  religious  innovation.49  The 
movement  is  divided  into  three  categories:  those  who  avoid  politics,  those  who  advocate 
in  favor  of  politics,  and  those  who  form  small  groups  of  jihadists.50  Salafism  is  often 
associated  with  Wahhabism,  even  though  the  two  ideologies  are  dissimilar.51 

In  the  fall  of  2011,  Salafis  began  smuggling  weapons  into  the  town  of  Darnmaj, 
despite  Houthi  requests  to  stop.  This  resulted  in  two  years  of  infighting  between  Sunni 
and  Shia  Muslims  in  Yemen.  The  Houthis  and  Salafis  perpetuated  the  conflict  until  a 
presidential  committee  brokered  a  ceasefire  on  January  11,  2014.  Part  of  the  deal 
involved  the  forced  relocation  of  hundreds  of  non-local  Salafi  families,  in  addition  to 
150,000  refugees  from  the  conflict  area.52 

46  U.S.  Government,  Department  of  Defense,  and  U.S.  Army,  The  Conflicts  in  Yemen  and  U.S. 
National  Security  -  Yemeni  Regional  Politics  and  Saudi  Arabia,  Drones,  Qat  Chewing,  Al-Qaeda,  War  on 
Terror,  Houthi  Tribesmen  Rebellion,  Zaydi  Shiite  Sect,  Kleptocracy  (Cleveland,  OH:  Progressive 
Management,  2014),  Kindle  edition. 

47  Joseph  J.  Hobbs,  Fundamentals  of  World  Regional  Geography,  ed.  4,  (Cengage  Learning,  2016), 

232. 

48  U.S.  Government,  Department  of  Defense,  U.S.  Army,  U.S.  Navy,  U.S.  Marine  Corps,  Army  War 
College,  and  U.S.  State  Department,  2015  Complete  Guide  to  Al-Qaeda  in  Yemen:  Al-Qaida  in  the  Arabian 
Peninsula  (AQAP)  (Cleveland,  OH:  Progressive  Management,  2015),  Kindle  edition. 

49  "Politics  and  the  Puritanical,”  Economist.  June  2015,  "http://www.economist.com/news/middle- 
east-and-africa/2 16561 89-islams-most-conservative-adherents-are-finding-politics-hard-it-beats. 

50  Ibid. 

51  Ibid. 

52  U.S.  Government  et  al..  The  Conflicts  in  Yemen  and  U.S.  National  Security,  232. 
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D.  AL-QAEDA  IN  THE  ARABIAN  PENINSULA  AND  YEMEN 

AQAP  has  systematically  built  their  network  in  Yemen  over  time,  starting  in  2006. 
By  2009,  AQAP  was  known  simply  as  Al-Qaeda  in  Yemen  (AQY)  and  had  grown  strong 
enough  to  announce  a  southern  independence  movement.  Bin  Laden  noticed  that  tribal 
support  was  imperative,  and  in  2000  sent  messengers  to  Yemen  to  win  local  support 
through  propaganda,  welfare  programs,  and  other  essential  material  needed  in 
impoverished  villages.53 

AQY  followed  a  systematic  process  for  every  village  they  entered.  They  began  by 
announcing  their  arrival  by  disseminating  propaganda  handbills,  CDs,  and  DVDs  which 
contained  their  dawa  ideology.  Slowly,  they  set  about  convincing  influence  leaders  in 
each  village  to  support  them.  Once  they  had  influenced  a  small  number  of  well-respected 
villagers  to  vouch  for  them,  they  would  leave  a  few  fighters  behind  and  move  on  to  the 
next  village.54  In  a  relatively  short  time,  they  had  exponentially  increased  domestic 
support  of  AQY  and  global  support  of  AQAP. 

AQY  also  used  other  relatively  ingenious  methods  to  expand  in  Yemen,  such  as 
marrying  into  a  given  village  and  planning  prison  breaks  for  members  who  were 
imprisoned.  Ultimately,  they  took  advantage  of  uneducated,  impoverished,  and 
disenfranchised  Yeminis  villages.55 

They  also  had  a  well-developed  communication  and  marketing  strategy  that 
enabled  them  to  build  connective  insurgent  tissue  in  Yemen  and  around  the  globe.  Their 
marketing  campaign  exponentially  increased  domestic  support  of  AQY  in  impoverished 
and  disenfranchised  Yemini  villages.56 


53  Abdel  Bari  Atwan,  After  Bin  Laden:  Al-Qaeda,  the  Next  Generation  (London,  UK:  Saqi  Books, 
London,  2012),  303. 

54  Ibid. 

55  Ibid. 

56  Ibid. 
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AQY  fighters  also  assisted  villagers  with  basic  security.  Villagers  in  the  Rada’a 
district,  A1  Bayda  Governorate,  were  seeking  police  or  military  basic  security,  which  they 
were  not  receiving  from  the  national  government.57 

Of  course,  not  everything  AQY  did  was  positive.  On  December  5,  2013,  AQY 
attacked  a  hospital,  leaving  over  50  people  dead  and  215  injured.  Seeing  the  negative 
reaction,  AQY  sent  an  apology  to  the  families  of  the  deceased  but  stated  that  though  the 
attack  had  been  an  accident,  compartments  in  the  building  had  supported  U.S.  drone 
operations.58 

E.  GOVERNMENT  OF  YEMEN 

President  Hadi  (Yemeni  president  February  2012-present)  has  generally  been 
associated  with  attempts  to  transform  Yemen  and  heal  its  deep  wounds.  His  greatest 
achievement  may  be  the  National  Dialogue  Conference  (NDC),  which  convened  in 
November  2013  and  concluded  in  January  2014.  Hadi  began  the  conference  by  returning 
land  in  southern  Yemen  seized  during  the  civil  war.59  During  the  conference,  he  proposed 
transfonning  Yemen  into  a  federal  system  with  six  regions.60  Hadi  believed  having  six 
regions  would  benefit  the  southern  Yemeni  cities,  his  base  of  support.  This  plan  did 
nothing  for  the  opposition,  however.  Houthis  living  in  the  poorest  northern  mountain 
region  of  Yemen  would  receive  minimal  financial  support  and  lose  their  sea  access.  Not 
surprisingly,  Hadi  was  unable  to  gain  the  support  he  needed  to  enact  his  federation 
proposal,  because  the  Houthis  saw  no  benefit  in  it.61 

In  a  strategic  sense,  the  NDC  was  an  initiative  to  bring  peace  to  Yemen,  yet  it  was 
immediately  marked  with  violence.  The  conference  started  with  an  assassination  of  a 


57  Ibid. 

58  Ibid. 

59  Ibid. 


60  U.S.  Government  et  al.,  2015  Complete  Guide  to  Al-Qaeda  in  Yemen. 

“Yemen  Crisis:  Who  Is  Fighting  Whom?”  BBC  News,  October  14,  2016,  http://www.bbc.com/ 
news/world-middle-east-293 1 9423 . 
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Houthi  representative  on  November  22,  20 13. 62  It  was  extended  two  weeks  past  its 
scheduled  conclusion  in  January  because  of  an  impasse  and  then  completely  collapsed 
with  the  assassination  of  a  second  Houthi  representative  on  January  21,  2014. 

Internationally,  Yemen  was  having  mixed  luck,  though  the  country  was  making 
progress  in  some  circles.  In  October  2013,  Hadi  met  with  EU  leaders  from  France  and 
Gennany  who  praised  the  NDC  initiative.63  On  the  other  hand,  Transparency 
International  ranked  Yemen  in  the  top  10  most  corrupt  states.64 

Domestically,  the  Yemeni  government  faces  serious  governance  challenges.  The 
government  approved  drone  strikes  and  army  attacks  on  VEOs,  which  were  likely  the 
cause  of  the  numerous  antigovernment  protests  that  followed.  Citizens  then  began 
demanding  human  rights  after  AQY  enacted  sharia  law  in  several  areas  they  had  siezed. 
Regions  and  tribes  that  were  unhappy  with  sharia  law  went  to  the  NDC  to  express  their 
views.  People  unhappy  with  police  corruption  protested  with  slogans  such  as  “Handcuffs 
Must  Break.”65  To  make  matters  worse,  Yemen  approved  several  U.S.  drones  strikes  that 
resulted  in  collateral  damage.  The  “Red  Wedding”  was  one  such  drone  strike.  On 
December  12,  2013,  15  people  were  killed  on  their  way  to  a  wedding  in  A1  Bayda. 
Tragically,  the  wedding  party  had  been  mistaken  for  an  AQAP  convoy.66  Yemen’s 
government  also  faced  issues  providing  basic  necessities  such  as  electricity,  low  energy 
and  food  prices,  and  security.  The  International  Institute  for  Strategic  Studies  Armed 
Conflict  reported  on  January  6,  2014,  that  54%  of  Yemenis  were  living  in  poverty  and 
40%  of  Yemenis  were  unemployed.67  Roads  and  travel  inside  much  of  the  country  have 
been  blocked  by  VEOs. 

62  International  Institute  for  Strategic  Studies,  “Yemen  (Houthis  /  AQAP  /  SMM),”  Armed  Conflict 
Database,  accessed  October  11,  2016,  http://acd.iiss.org/en/conflicts/yemen—houthis-aqap-smm- 
9651?year=2013&month=10. 

63  Ibid. 

64  Transparency  International,  “Corruption  Perceptions  Index  2013,”  accessed  June  14,  2016, 
http://www.transparency.org/cpi2013/results. 

65  Ibid. 

66  Ibid. 

67  International  Institute  for  Strategic  Studies,  “Yemen  (Houthis  /  AQAP  /  SMM),”  Armed  Conflict 
Database,  accessed  October  11,  2016,  http://acd.iiss.org/en/conflicts/yemen— houthis-aqap-smm- 

965 1  ?year=20 1 3  &month=  10 
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IV.  RESEARCH  METHODS 


A.  HYPOTHESIS 

We  propose  that  social  media  analysis  can  overcome  the  limitations  of  traditional 
collection  and  assessment  methods  by  assessing  a  population’s  attitudes,  beliefs,  and 
opinions  using  social  media  data  and  sentiment  analysis.  If  relationship  is  found  via 
logistical  regression  analysis,  the  study  may  provide  scholars,  practitioners,  and  the  U.S. 
military  a  new  method  to  analyze  sentiment  in  lieu  of  a  traditional  survey  and  a  way  to 
assess  the  full  spectrum  of  government  operations  and  lines  of  effort  in  conflict  zones. 
Additionally,  future  research  may  provide  insight  into  social  network  analysis,  social 
movement  theory,  and  communications  theory. 

B.  DATA  AND  METHODS 

1.  Social  Media 

We  began  our  research  with  an  archive  of  Twitter  data  that  contained  three  billion 
messages  and  40  terabytes  of  data  in  total,  licensed  by  NPS.  The  archive  represented  a 
10%  randomized  sample  of  the  total  number  of  tweets  that  occurred  over  a  12-month 
timeframe.  Our  research  analyzed  data  spanning  a  four-month  timeframe,  from  October 
1,  2013,  through  January  31,  2014. 

Traditional  computational  approaches  search  across  strings  of  text  in  serial  order. 
However,  searching  40  terabytes  of  Twitter  traffic  using  standard  computational 
approaches  would  be  very  cumbersome  and  slow.  Our  approach  instead  utilized  a  highly 
parallelized  in-memory  database  application  developed  by  Camber  Warren  to  speed  the 
computations.68  The  approach  utilizes  “swarms”  of  parallel  computational  units  to 
operate  in  tandem.  This  requires  parallel  loading  of  individual  CPU  cores,  allowing  it  to 


68  T.  Camber  Warren,  “Mapping  the  Rhetoric  of  Violence:  Political  Conflict  Discourse  and  the 
Emergence  of  Identity  Radicalization  in  Nigerian  Social  Media,”  prepared  for  presentation  at  the  Annual 
Meeting  of  the  American  Political  Science  Association,  September  3-6,  2015,  San  Francisco,  CA. 
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avoid  “resource  conflicts,  without  the  need  for  hierarchical  control  structures,”  which  are 
a  traditional  computational  bottleneck.69 

The  first  task  using  this  approach  is  to  reference  each  message  to  a  location  in 
time  and  space.  Each  tweet  contains  underlying  metadata  that  identifies  the  tweet’s  date 
time  stamp  and  the  user’s  location.  The  user  location  is  based  on  either  the  user’s  profile 
setting  for  their  hometown  or,  less  commonly,  geo-tagging  on  the  tweet  itself.  The 
location  is  converted  into  latitude/longitude,  allowing  us  to  match  it  to  other  geo- 
referenced  data  sources. 

2.  Sentiment  Dictionary 

Our  next  task  was  to  develop  the  parameters  of  our  message  content  search,  by 
building  a  “sentiment  dictionary”  (sometimes  called  a  “sentiment  lexicon”).  This  is  a  list 
of  words  that  are  assigned  positive,  negative,  and/or  neutral  values  manually  or  through 
natural  language  processing  (NLP).  We  chose  the  AFINN  dictionary  that  was  developed 
by  Finn  Arup  Nielsen.70  One  advantage  of  this  particular  lexicon  is  that  it  explicitly 
captures  intensity,  using  a  scale  that  ranges  from  -5  to  +5,  allowing  researchers  to 
eliminate  neutral  responses  and  focus  on  extreme  negative  or  extreme  positive  responses. 
We  limited  our  lexicon  to  words  with  score  magnitudes  of  at  least  3  (positive  or  negative) 
in  order  to  reduce  neutral  responses  and  limit  analysis  to  the  far  left  and  right  sides  of  the 
response  scale. 

We  also  selected  concept  search  terms  reflecting  key  aspects  of  Yemeni  society 
and  politics.  First,  we  sought  to  capture  the  concept  of  institutional  confidence.  The 
naming  convention  for  search  terms  associated  with  government  support  was  designed  to 
include  formal  and  infonnal  ways  of  referencing  government  institutions: 

1)  Yemeni  +  name  of  institution/individual  (e.g.,  Yemeni  Army,  Yemeni  Soldiers), 

2)  our  +  name  of  institution/individual  (e.g.,  our  government,  our  president), 


69  Ibid. 

70  Finn  Amp  Nielsen,  “A  new  ANEW:  Evaluation  of  a  word  list  for  sentiment  analysis  in 
microblogs  ”  arXiv  preprint  arXiv:  1103.2903  (2011). 
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3)  my  +  name  of  institution/individual  (e.g.,  my  city  council,  my  mayor), 

4)  location  +  name  of  institution/individual  (e.g.,  Aden’s  City  Council,  Aden’s  mayor), 

5)  title  +  name  of  senior  individual  associated  with  an  institution  (e.g.,  President  Hadi), 

6)  name  of  senior  individual  associated  with  an  institution  without  title  (e.g.,  Abdullah 
Mohsen  al-Akwa). 

Second,  we  sought  to  capture  the  concept  of  support  for  extremism.  The  naming 
convention  for  search  terms  associated  with  extremist  support  included  the  different  ways 
the  organization  was  identified  along  with  senior  leaders  within  the  organization.  The 
names  and  leaders  were  derived  from  government  websites  and  multiple  conflict 
databases.  A  list  of  search  terms  was  compiled  to  separately  assess  each  of  the  10 
institutions  and  the  violent  extremist  organizations  evaluated  in  the  survey.  The 
combination  of  all  tenns  associated  with  the  government  comprised  the  search  criteria  for 
institution.  All  terms  associated  with  VEOs,  along  with  specific  terms  related  to  jihad  and 
sharia,  comprised  the  search  criteria  for  extremism.71 

Both  the  sentiment  dictionary  and  concept  dictionary  were  then  translated  from 
English  to  Arabic  using  the  automated  Google  Translate  API.  In  an  attempt  to  address  the 
impact  of  translation  potentially  switching  a  word’s  sentiment,  we  also  translated  each 
word  back  into  English  and  replaced  words  that  no  longer  matched  their  originals  with 
manually  selected  words  from  Google  Translate  or  else  deleted  them.  We  checked  the 
Google  Translate  results  with  two  humans.  The  first  was  Major  Joshua  Wines,  and  the 
second  was  a  native  Arabic  speaker  who  teaches  Arabic  at  the  U.S.  Defense  Language 
Institute  in  Monterey,  California. 

3.  Kernel  Density  Estimates 

Traditional  spatial  regression  techniques  entail  selecting  a  political  boundary  like 
a  city,  county,  or  state,  as  the  unit  of  analysis.  However,  such  approaches  are  subject  to  a 


71  Concept  dictionary  is  in  attached  Appendix:  Concept  Dictionary. 
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statistical  bias  called  the  modifiable  areal  unit  problem  (MAUP),72  which  renders  such 
estimates  highly  unreliable.  In  contrast,  the  approach  taken  here  seeks  to  convert  the 
metrics  derived  from  social  media  messages  into  continuous  spatial  surfaces.  For  each 
concept  and  for  each  month  in  our  study  period,  we  estimated  a  continuous  spatial 
surface,  representing  the  relative  density  of  messages  referencing  a  concept  in  a  particular 
place  and  time.  We  smoothed  the  results  using  two-dimensional  binned  Gaussian  kernel 
density  interpolation.73  To  avoid  arbitrary  limitations  on  the  spatial  ranges  of  the 
estimated  effects,  we  generated  densities  by  taking  the  average  across  multiple  kernel 
density  bandwidths,  ranging  from  1  kilometer  to  500  kilometers.  The  result  is  a  heat  map 
representing  the  relative  density  of  messages  about  a  particular  sentiment  and  concept,  as 
illustrated  in  Figure  1,  which  shows  the  negativity  toward  the  National  Government 
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-  1 


-  0.1 


0.01 


Figure  1.  Heat  Map.  National  Government  Negative  Tweets.  January  20 14. 74 


72  Stan  Openshaw  and  S.  Openshaw,  The  Modifiable  Areal  Unit  Problem  (Geo  Abstracts  University  of 
East  Anglia,  1984). 

73  T.  Camber  Warren,  “Mapping  the  Rhetoric  of  Violence:  Political  Conflict  Discourse  and  the 
Emergence  of  Identity  Radicalization  in  Nigerian  Social  Media,”  prepared  for  presentation  at  the  Annual 
Meeting  of  the  American  Political  Science  Association,  September  3-6,  2015,  San  Francisco,  CA. 

74  Adapted  from  Twitter,  unpublished  data. 
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4.  Dependent  Variables 


To  attempt  to  validate  the  inferences  derived  from  our  new  social  media  metrics, 
we  rely  on  existing  opinion  poll  data,  collected  by  Gallup  between  November  14,  2013, 
and  January  17,  2014,  under  the  Global  Research  and  Assessment  Program  (GRAP)  for 
United  States  Special  Operations  Command  (USSOCOM)  J39.  The  poll  consisted  of 
9,450  civilian  Yemeni  adults  aged  16-65  with  a  sampling  based  on  a  stratified, 
multistage  cluster  design.  The  key  advantage  of  this  data  source  is  that  it  provides 
anonymous  measurements  of  political  support  for  the  Yemeni  state  and  various  insurgent 
groups,  which  can  be  geo-referenced  to  specific  cities  and  towns,  allowing  researchers  to 
generate  an  aggregate  spatial  representation  of  the  sentiments  recorded  by  the  poll.75 

All  dependent  variables  were  coded  dichotomously. 

Extremism  Support  and  Extremism  Opposition  are  derived  from  the 
Extremism  index,  which  ranges  from  one  to  four  and  is  a  compilation  of  25  questions 
concerning  violent  extremist  organizations  (VEOs),  attitudes  toward  sharia,  and  jihad, 
with  a  Cronbach’s  alpha  score  of  0.62. 

Institutional  Support  and  Institutional  Opposition  are  derived  from  the 
Institutional  Confidence  index,  which  is  a  compilation  of  10  questions  with  a  Cronbach’s 
alpha  score  of  0.91.  This  index  included  questions  about  confidence  in  national 
government,  state/provincial  government,  local/city  government,  military,  police/law 
enforcement,  financial  institutions/banks,  the  national  courts,  sharia  courts,  tribal  justice, 
and  the  electoral  system. 

Al-Qaeda  Support  and  Al-Qaeda  Opposition  counted  any  responses  indicating 
that  either  al-Qaeda’s  or  AQAP’s  influence  had  a  very  positive  influence  as  support. 
Similarly,  any  responses  indicating  these  organizations  had  a  very  negative  influence 
were  counted  as  an  opposition. 


75  Jones,  Counterinsurgency  in  Afghanistan. 
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National  Government  Support  and  National  Government  Opposition  are 

derived  by  counting  responses  indicating  confidence  in  the  national  government,  with 
indications  of  no  confidence  used  as  indicator  of  opposition  to  the  national  government. 

5.  Independent  Variables 

The  independent  variables  are  based  on  the  heat  maps  generated  from  the 
sentiment  and  concept  dictionary  results.  They  change  depending  upon  the  evaluated 
dependent  variable.  Additionally,  each  independent  variable  forms  the  basis  of  a  model 
set.  These  sets  are  evaluated  in  three  different  ways  with  demographic  control  variables, 
social  media  variables,  or  a  combination  of  the  two.  We  annotate  these  as 
DEMOGRAPHIC  VARIANCE,  TWITTER  VARIANCE,  and  COMBINATION 
VARIANCE. 

The  independent  variable  extremism  is  the  result  of  all  words  associated  with 
VEOs  and  jihadist  ideology  as  noted  in  the  Sentiment  Dictionary  section.  The 
independent  variables  are  divided  between  either  EXTREMISM  POSITIVE ,  which  are 
based  on  the  positive  sentiment  dictionary,  or  EXTREMISM  NEGATIVE,  which  is 
based  on  the  negative  sentiment  dictionary.  It  is  further  broken  down  MONTHLY  from 
October  to  January,  and  a  TOTAL  of  all  four  months  for  a  total  of  10  extremism 
independent  variables.  Along  the  same  lines  each  single  organization  was  evaluated.  For 
instance,  in  evaluating  support  for  al-Qaeda,  there  is  an  AQ  POSITIVE  and  an  AQ 
NEGATIVE  independent  variable  for  each  month,  and  a  TOTAL  of  all  four  months  for  a 
total  of  10  AQ  independent  variables. 

The  independent  variable  of  INSTITUTION  is  the  result  of  all  words  associated 
with  institutions  noted  in  the  Sentiment  Dictionary  section.  The  institution  variables  are 
divided  like  the  extremism  variables.  The  institution  variables  are  divided  into  the 
variables  INSTITUTION  POSITIVE  and  INSTITUTION  NEGATIVE.  Insitution  is 
further  broken  down  into  individual  months,  called  MONTHLY  from  October  to  January 
with  a  TOTAL  of  all  months  for  a  combined  10  institution  independent  variables.  Along 
the  same  line,  the  national  government  was  evaluated.  There  is  a  NATIONAL 
GOVERNMENT  POSITIVE  and  a  NATIONAL  GOVERNMENT  NEGATIVE 
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independent  variable  for  most  months,  and  a  TOTAL  of  all  the  months  for  a  total  of  9 
national  government  independent  variables.  There  are  fewer  national  government 
independent  variables  because  there  were  not  enough  Twitter  results  to  generate  a 
positive  heat  map  for  the  month  of  October  for  this  search  category. 

6.  Control  Variables 

Most  of  the  control  variables  are  based  on  demographic  information.  We  did  this 
to  address  a  common  critique  against  using  social  media  data,  which  is  that  social  media 
users  are  not  a  representative  sampling  of  the  population.  Some  may  be  concerned  that  if 
we  find  a  significant  relationship  that  appears  to  be  driven  by  social  media  data,  it  may 
actually  reflect  demographic  factors.  Therefore,  we  controlled  for  a  series  of 
demographic  factors  that  are  known  to  influence  political  attitudes:  Internet  access, 
urban,  gender,  age,  education,  and  wealth.  All  control  variables  were  coded 
dichotomously. 

The  other  control  variables  are  derived  from  the  Twitter  density  estimates.  We 
controlled  for  both  the  overall  number  of  tweets  and  the  total  number  of  tweets  about  a 
concept  in  a  specific  timeframe.  We  did  this  to  account  for  the  influence  of  total  message 
volume  so  it  could  be  differentiated  from  message  content  (e.g.,  sentiment).  We  also 
controlled  for 

Internet  assigns  a  value  of  1  to  any  use  of  the  Internet,  Urban  is  coded  as  1  for 
urban  areas  with  a  population  of  at  least  5,000  inhabitants.  Youth  counts  those  24  years 
of  age  or  younger,  Educated  counts  those  with  at  least  some  university  education.  This 
variable  was  dichotomously  coded  with  those  with  at  least  some  university  education  as  1 
and  everyone  else  as  0. 

Wealthy  -  This  control  variable  was  derived  from  the  Gallup  Poll.  It  reflects 
Yemenis  with  an  income  of  60,000  Yemeni  Riyals  (YER)  per  year  or  higher.  Anyone 
earning  at  least  60,000  YER  was  dichotomously  coded  as  1  and  those  below  that  amount 
as  0. 

We  also  control  for  background  spatial  characteristics: 
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Population  Density  -  This  control  variable  is  derived  from  a  population  density 
heat  map  and  reflects  a  more  refined  impact  of  population  in  the  urban  areas. 

Total  Tweets  -  This  control  variable  is  derived  from  the  combination  of  heat 
maps  for  each  month  from  October  2013  through  January  2014.  It  is  the  sum  of  all  tweets 
originating  from  Yemen  during  this  timeframe.  This  variable  was  log  transformed  in 
order  to  account  for  its  heavy  tailed  distribution. 

Total  Concept  Tweets  -  This  control  variable  is  based  on  the  combination  of  heat 
maps  that  relate  to  each  of  the  independent  variables.  It  encompasses  the  months  of 
October  2013  to  January  2014.  It  is  the  sum  of  all  tweets  that  relate  to  a  specific  topic 
(e.g.,  extremism,  national  government)  during  the  specified  time.  This  variable  was  also 
log  transformed  in  order  to  account  for  its  heavy  tailed  distribution. 

Provincial  Boundaries  -  This  control  variable  is  based  on  the  20  provincial 
political  boundaries  and  the  capital  city  boundary  within  Yemen.  These  were  included  as 
fixed  effects  in  the  models. 

C.  REGRESSION  ANALYSIS 

We  used  a  logistic  regression  for  our  analysis.  Logistic  regressions  are  a  statistical 
method  in  which  the  dependent  variable  is  dichotomous,  also  known  as  logit  regressions, 
or  the  logit  model.  Dichotomous  dependent  variables  can  take  only  two  values,  such  as 
one/zero,  win/lose,  or  alive/dead.  The  logistic  model  is  used  to  estimate  the  probability  of 
a  binary  response  based  on  one  or  more  independent  variables.76 

In  summary,  we  assessed  eight  dependent  variables  that  were  derived  from  existing 
opinion  polls.  The  dependent  variables  focus  on  support  for  or  opposition  to  the 
government  and  support  for  or  opposition  to  the  insurgency.  After  developing  the 
dependent  variables  we  built  39  independent  variables  that  derived  from  positive  and 
negative  Twitter  sentiment.  Finally,  we  controlled  for  a  variety  of  factors  related  to  the 
dependent  and  independent  variables.  In  the  end,  we  experimented  and  evaluated  117 
different  models. 

76  Cox,  “The  Regression  Analysis  of  Binary  Sequences.” 
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V.  RESULTS 


Taken  as  whole,  these  statistical  models  generate  five  major  categories  of  results. 
First,  we  find  that  the  addition  of  social  media  metrics  consistently  improves  the  level  of 
predictive  success  when  assessing  popular  sentiments,  over  and  above  the  predictions 
generated  by  demographic  patterns  alone.  Second,  we  find  that  using  our  method,  we  are 
able  to  generate  estimates  of  spatial  patterns  of  sentiment  that  strongly  resemble  estimates 
derived  from  traditional  polling  methods.  Third,  it  seems  that  this  congruence  is 
strongest  when  our  method  was  used  to  identify  oppositional  sentiments,  as  opposed  to 
supportive  sentiments.  Fourth,  our  methods  yield  better  and  more  consistent  results  when 
examining  VEO’s  and  extremism,  than  when  examining  support  for  the  government. 
Finally,  when  we  examine  data  by  month,  even  controlling  for  background  spatial 
patterns  and  demographic  patterns,  we  find  that  the  social  media  data  still  provides  an 
important  improvement  in  the  predictive  success  of  the  models. 

A.  FINDING  ONE  -  IMPROVES  PREDICTIONS 

The  most  significant  and  reliable  finding  is  that  Twitter-derived  sentiment  metrics 
strengthens  the  predictive  success  of  the  statistical  models.  Every  model  with  Twitter  and 
demographic  data  combined  resulted  in  better  AIC  scores  than  those  models  with  either 
demographic  or  Twitter  data  by  themselves.  Why  is  this  important?  The  AIC  score  is 
basically  a  comparison  tool  that  helps  us  evaluate  the  predictive  success  of  each  model.77 
The  AIC  measures  how  well  a  specific  model  fits  a  dataset,  which  in  this  case  are 
estimates  of  political  support  and  opposition.  It  also  considers  the  number  of  variables, 
penalizing  those  models  that  are  so  flexible  that  can  just  as  easily  fit  any  dataset.  Most 
importantly  the  AIC  score  is  an  indicator  of  confidence  in  a  model. 

When  examining  the  findings  presented  in  Table  1,  it  is  important  to  note  that  the 
lower  the  AIC  score  is,  the  more  confidence  we  should  have  in  the  model.  These  eight 
model  sets  are  those  with  the  TOTAL  independent  variables.  We  believe  that  these 

77  Hirotugu  Akaike,  “A  new  look  at  the  statistical  model  identification.”  IEEE  transactions  on 
automatic  control  19,  no.  6  (1974). 
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particular  models  are  the  most  reliable,  because  they  encompass  Twitter  data  for  the  same 
time  period  that  the  original  opinion  poll  was  conducted.  The  results  on  the  far  right  show 
the  AIC  scores  for  each  of  the  combined  models,  supporting  our  claim  of  improved 
predictive  success  across  the  board. 


Table  1.  Akaike  Information  Criterion. 


DEPENDENT  VARIABLES 

Demographics 

Twitter 

Combined 

Al  Qaeda  Support 

612.572 

649.248 

541.813 

Al  Qaeda  Opposition 

9,423.769 

9,700.528 

8,134.125 

Extremism  Support 

5,144.949 

5,387.780 

4,299.991 

Extremism  Opposition 

3,103.926 

3,483.122 

2,857.858 

National  Govt  Support 

6,592.986 

6,443.039 

5,764.233 

National  Govt  Opposition 

9,479.310 

11,007.990 

8,754.131 

Institution  Support 

8,459.854 

8,322.456 

7,297.280 

Institution  Opposition 

9,914.237 

10,735.100 

8,576.029 

B.  FINDING  TWO  -  SIMILARITY  OF  SPATIAL  PATTERNS 

Closely  related  to  the  first  finding  is  that  the  estimates  derived  from  social  media 
generate  similar  patterns  of  inferences  to  those  generated  by  traditional  polling  data. 
There  are  two  ways  to  demonstrate  this  finding.  The  first  is  a  statistical  approach  and  the 
other  is  a  visual  depiction.  Statistically,  30  out  of  the  39  TWITTER  VARIANCE  models 
show  a  better  AIC  than  the  DEMOGRAPHIC  VARIANCE  models.  The  only  model  sets 
that  do  not  show  that  the  TWITTER  VARIANCE  is  better  are  those  with  National 
Government  Support  and  Institutional  Support  dependent  variables. 

The  other  way  to  demonstrate  this  is  through  the  visual  depiction  in  Figure  2, 
which  shows  a  comparison  of  polling  and  twitter  sentiment  by  province.  The  heat  map  on 
the  left  is  the  mean  of  all  polling  responses  concerning  AQ  and  AQAP  by  province.  The 
darker  the  color  means  there  is  a  greater  the  level  of  opposition.  The  heat  map  on  the 
right  is  the  mean  of  all  negative  tweets  about  AQ  and  AQAP  from  October  to  January  by 
province.  The  darker  color  is  there  is  a  greater  amount  of  negative  tweets.  As  you  can  see 
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from  the  maps,  the  majority  of  the  provinces  are  closely  aligned  with  the  exception  of 
eastern  Yemen,  an  extremely  rural  region  with  little  to  no  Internet  access. 


Can  social  media  algorithms 
replicate  survey  results? 


Gallup  Survey  Response 


‘Are  you  opposed  to  al-Gaeda?” 


Twitter  Data  Algorithm  Result 


“Are  you  opposed  to  al-Gaeda?’ 


\ 
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■  [3.8,3. 9) 

■  [3.9,4) 


Country:  Yemen 


"Empty  Quarter” 
Largest  expanse  of 
sand  on  planet  = 
little  Twitter 


<Y> 


[4,5) 

■  [5,6) 

■  16,7) 

■  [7,8) 

■  [8.9) 

■  [9,10) 

■  [10,11) 


Authors:  Andrew  Bourret,  Joshua  Wines.  Jason  Mendes 


Figure  2.  Comparing  Survey  Responses  and  Twitter  Metrics  Indicating 

Opposition  to  al-Qaeda.78 


This  is  not  to  assert  that  our  research  finds  that  twitter  sentiment  can  be  used  in 
lieu  of  polling.  There  are  too  many  inconsistencies  between  the  different  models  to 
definitively  make  such  a  claim.  The  purpose  is  to  highlight  the  competitiveness  between 
the  impact  of  twitter  data  and  demographic  data  on  polling  sentiment.  It  is  simply  another 
step  in  confirming  or  denying  the  value  of  social  media  as  a  source  of  data-driven 
insights. 


78  Adapted  from  Gallup  survey,  unpublished  data;  Twitter,  unpublished  data. 
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c. 


FINDING  THREE  -  SENTIMENT  MATTERS 


Interestingly,  our  results  also  indicate  that  Twitter  is  more  suitable  for  picking  up 
certain  types  of  information.  For  example,  the  results  in  Table  2  indicate  that  negative 
sentiment  may  be  more  reliable.  These  results  are  the  only  TOTAL  independent  variables 
assessed  using  the  Combination  Variance  that  had  statistically  significant  coefficients. 
Just  as  importantly,  they  show  a  positive  relationship  between  polling  and  twitter 
sentiment.  As  mentioned  earlier  in  Finding  One,  we  have  the  greatest  confidence  in  the 
TOTAL  independent  variables  and  the  Combination  Variance  always  has  the  best  AIC. 
These  relationships  can  be  found  looking  at  1)  the  AQ  Negative  Sentiment  row  under  the 
al-Qaeda  Opposition  combined  column;  2)  the  Extremism  Negative  sentiment  row  under 
the  Extremism  Opposition  combined  column. 


Table  2.  Combined  Negative  Sentiment. 


Al 

Demographics 


Qaeda  Opposition 

Twitter  Combined 


Extremism  Opposition 

Demographics  Twitter  Combined 


Internet  Users 

0.317'"  (0.122) 

0.254*  (0.141)  -0.127(0.201) 

-0.158(0.246) 

Urban  Residents 

0.038  (0.073) 

0. 148*  (0.086)  -0.062  (0. 146) 

-0.033  (0.167) 

Men 

1.226'"  (0.054) 

1.271*"  (0.058)  -0.564*’*  (0.102) 

-0.750*"  (0.1 17) 

Youth 

0.087  (0.056) 

0.052(0.061)  0.259**  (0.104) 

0.176(0.116) 

Educated 

0.537"’ (0.111) 

0.605*’*  (0.125)  0.047(0.162) 

0.242(0.183) 

Wealthy 

0.391*"  (0.066) 

0.386***  (0.073)  0.138  (0.121) 

0.083  (0.134) 

Population  Density 

0.016(0.034) 

-0.022  (0.039)  -0.080  (0.063) 

-0.120  (0.077) 

AQ  Negative  Sentiment 

2.012*" 

(0.235)  0.799***  (0.305) 

Total  AQ  Tweets 

-1.281*" 

(0.328) -1.089*’ (0.428) 

Extremism  Negative  Sentiment 

2.405** 

(0.332) 

0.822*  (0.486) 

Total  Extremism  Tweets 

-2.409* 

(0.632) 

0.208  (0.753) 

Total  Tweets 

-0.132*” 

(0.036)  -0.027(0.062)  -0.406* 

(0.084) 

-0.093  (0.148) 

Constant 

-0.191  (0.227)  -1.764*" 

(0.408)  0.163(0.599)  -2.535***  (0.446) -5.260* 

(1.030)  -4.759*"  (1.541) 

Province  fixed  effects 
AIC 


NA 

9700.528 


Included 

9423.769 


Included 

8134.125 


NA 

3483.122 


Included 

3103.926 


Included 

2857.858 


p<.01;  p<.05;  p<.l 


Columns  are  comprised  of  two  values.  Numbers  on  the  left  side  of  each  column  are  coefficient 
values.  Numbers  to  the  right  side  of  each  column  are  standard  error. 


Even  in  the  negative  Total  independent  variables  with  Twitter  Variance,  only  one 
model  with  a  statistically  significant  coefficient  showed  a  negative  relationship  (Table  3). 
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This  reinforces  the  finding  that  sentiment  is  potentially  better  at  evaluating  negative 
sentiment. 


Table  3.  Significance  in  Expected  Direction  (Total). 


DEPENDENT  VARIABLES 

Expected 

Unexpected 

Insignificant 

A1  Qaeda  Support 

X 

Al  Qaeda  Opposition 

X 

Extremism  Support 

X 

Extremism  Opposition 

X 

National  Govt  Support 

X 

National  Govt  Opposition 

X 

Institution  Support 

X 

Institution  Opposition 

X 

When  we  looked  at  the  reliability  of  positive  sentiment,  we  found  some  additional 
supporting  arguments.  For  example,  there  was  only  one  other  TOTAL  independent 
variable  model  we  assessed  using  the  COMBINATION  VARIANCE  that  had  statistically 
significant  results.  This  was  the  National  Government  Support  model,  but  it  had  a 
negative  relationship  between  twitter  and  polling  (reference  Table  4).  This  result  along 
with  the  negative  relationship  found  in  the  Institutional  Support  TWITTER  VARIANCE 
model  seems  to  reinforce  the  finding  that  negative  sentiment  is  more  reliable.  We  infer 
that  Twitter  data  is  a  reliable  source  of  ground  truth  when  there  is  a  positive  relationship 
between  positive  twitter  sentiment  /  polling  support  and  negative  twitter  sentiment  / 
polling  opposition.  Based  on  this  analysis,  it  seems  like  those  that  spoke  negatively  on 
twitter  were  likely  to  be  in  the  same  geographic  space  as  those  that  spoke  negatively  in 
the  survey. 
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Table  4.  Combined  Sentiment. 


National  Government  Support  Institutional  Support 


Demographics 

Twitter 

Combined 

Demographics 

Twitter 

Combined 

Internet  Users 

-0.338”  (0.158) 

-0.419**  (0.175) 

-0.205  (0.125) 

-0.283**  (0.143) 

Urban  Residents 

-0.292*”  (0.101) 

-0.275”  (0.1 14) 

-0.169**  (0.082) 

-0.161*  (0.092) 

Men 

-0.277*”  (0.067) 

-0.337***  (0.071) 

-0.602*”  (0.057) 

-0.612*”  (0.061) 

Youth 

0.059  (0.074) 

0.098  (0.077) 

0.121*  (0.062) 

0.186***  (0.065) 

Educated 

-0.166(0.129) 

-0.145  (0.137) 

-0.237*’ (0.109) 

-0.208*  (0.118) 

Wealthy 

0.188**  (0.080) 

0.199*’ (0.086) 

0.039  (0.070) 

-0.002  (0.075) 

Population  Density 

0.019(0.045) 

0.086  (0.054) 

-0.057  (0.038) 

-0.023  (0.044) 

Natl  Govt  Positive  Sentiment 

-0.665*“  (0.102) 

-0.491***  (0.145) 

Total  Natl  Govt  Tweets 

0.342**’  (0.130) 

0.247  (0.253) 

Institution  Positive  Sentiment 

-1.324**' 

‘(0.351) 

-0.056  (0.443) 

Total  Institution  Tweets 

0.694** 

(0.338) 

-0.096  (0.475) 

Total  Tweets 

0.077*  (0.046) 

-0.249***  (0.087) 

0.451*** 

(0.035) 

-0.174***  (0.060) 

Constant 

-2.271*”  (0.315) 

-0.729*  (0.392) 

-3.346***  (0.797) 

-0.752***  (0.254) 

2.546*** 

(0.290) 

-1.333”  (0.570) 

Province  fixed  effects 

NA 

Included 

Included 

NA 

Included 

Included 

AIC 

6443.039 

6592.986 

5764.233 

8322.456 

8459 

'.854 

7297.280 

*p<.01;  p<.05;  p<.l 


Columns  are  comprised  of  two  values.  Numbers  on  the  left  side  of  each  column  are  coefficient 
values.  Numbers  to  the  right  side  of  each  column  are  standard  error. 


If  true,  this  may  mean  that  the  negative  sentiment  dictionary  is  perhaps  better  due 
to  translation.  However,  previous  research  shows  that  sentiment  dictionaries  that  are 
machine  translated  perform  only  slightly  worse  than  manually  created  dictionaries.79  The 
other  possible  explanation  is  that  Yemeni  Twitter  users  that  express  negative  sentiment 
are  a  more  representative  sample  of  the  population.  This  would  be  consistent  with  other 
research  that  states  social  media  in  the  Middle  East  functions  as  a  “free  space”  for 
criticism. 

D.  FINDING  FOUR  -  THE  IMPORTANCE  OF  THE  TOPIC 

The  story  does  not  end  there.  Some  of  the  other  results  challenge  the  third  finding. 
For  example,  every  extremism  and  AQ  related  model  with  MONTHLY  independent 
variables  (e.g.  October,  November,  etc.)  using  the  COMBINATION  VARIANCE  or 
TWITTER  VARIANCE  results  in  statistically  significant  coefficients  and  a  positive 
relationship.  This  is  regardless  of  whether  the  independent  variable  is  based  on  either 

79  Saif  M.  Mohammad,  Mohammad  Salameh,  and  Svetlana  Kiritchenko,  "How  Translation  Alters 
Sentiment  ”  Journal  of  Artificial  Intelligence  Research  1  (2015):  95-130. 
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positive  or  negative  sentiment.  In  contrast,  only  the  National  Government  Support  and 
the  Institutional  Support  models  have  relationships  in  both  the  expected  and  unexpected 
direction.  This  contrast  is  seen  in  Table  5,  which  distills  the  results  of  all  of  the 
MONTHLY  independent  variables  using  either  the  COMBINATION  VARIANCE  or 
TWITTER  VARIANCE.  It  captures  any  MONTHLY  statistically  significant  coefficient 
that  has  a  positive  or  negative  relationship  between  twitter  and  polling.  As  you  can  tell,  it 
seems  that  certain  concepts  are  easier  or  more  feasible  to  evaluate. 


Table  5.  Significance  in  Expected  Direction  (Any). 


DEPENDENT  VARIABLES 

Expected 

Unexpected 

A1  Qaeda  Support 

X 

At  Qaeda  Opposition 

X 

Extremism  Support 

X 

Extremism  Opposition 

X 

National  Govt  Support 

X 

National  Govt  Opposition 

X 

X 

Institution  Support 

X 

X 

Institution  Opposition 

X 

X 

These  inconsistencies  point  to  the  possibility  that  the  dictionary  didn’t  adequately 
capture  the  concepts  of  national  government  and/or  the  institution  index.  However,  there 
were  51  separate  terms  associated  with  national  government  and  890  separate  terms 
associated  with  the  institution  index.  It’s  unlikely  the  inconsistencies  are  due  to  a 
deficient  dictionary.  It’s  also  possible  that  individuals  that  tweet  negatively  about  the 
government  are  less  likely  to  self-identify  their  location  in  their  profile  out  of  fear. 
However,  there  are  negative  relationships  even  among  those  that  tweet  positively  about 
the  govermnent  and  positive  relationships  among  those  that  tweet  positively  about  AQ. 
Fear  of  the  government  does  not  seem  to  fit  as  an  answer.  It  may  simply  be  that  it’s  more 
difficult  to  evaluate  concepts  regarding  the  government  compared  to  AQ  and  extremism. 
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E. 


FINDING  FIVE  -  POTENTIAF  TO  EVAFUATE  BY  MONTH 


Lastly,  we  evaluated  each  dependent  variable  using  the  MONTHLY  positive  and 
negative  independent  variables.  This  was  the  case  for  each  model  except  for  NATIONAL 
GOVERNMENT  POSITIVE  sentiment.  As  noted  earlier,  there  weren’t  enough  Twitter 
data  points  to  make  a  heat  map  for  the  month  of  October.  The  value  of  running  so  many 
models  with  different  independent  variables  allowed  us  to  examine  the  reliability  of 
monthly  data.  In  fact,  we  looked  at  35  model  sets  of  MONTHLY  independent  variables. 

These  models  indicate  that  monthly  Twitter  sentiment  may  also  be  reliable.  We 
stated  in  Finding  One  that  we  have  the  greatest  confidence  in  the  TOTAL  independent 
variables.  However,  even  the  majority  of  the  MONTHLY  independent  variable  produces 
results  with  statistically  significant  coefficients  in  the  expected  direction.  Table  6 
captures  the  number  of  the  MONTHLY  TWITTER  VARIANCE  models  with  statistically 
significant  coefficients.  Out  of  those  27  models,  18  of  them  show  a  positive  relationship. 
These  statistical  outputs  are  important,  because  they  influence  our  confidence  level  in  the 
reliability  of  the  data.  While  these  results  aren’t  as  statistically  strong  as  the  previous 
findings,  they  do  indicate  there  is  a  significant  relationship  between  monthly  Twitter  data 
and  polling  results. 


Table  6.  Significance  in  Expected  Direction  (Monthly). 


DEPENDENT  VARIABLES 

October 

November 

December 

January 

Al  Qaeda  Support 

Expected 

Expected 

Al  Qaeda  Opposition 

Expected 

Expected 

Expected 

Extremism  Support 

Expected 

Expected 

Expected 

Expected 

Extremism  Opposition 

Expected 

Expected 

Expected 

National  Govt  Support 

Unexpected 

Unexpected 

Unexpected 

Unexpected 

National  Govt  Opposition 

Unexpected 

Expected 

Unexpected 

Expected 

Institution  Support 

Expected 

Expected 

Unexpected 

Institution  Opposition 

Expected 

Unexpected 

Unexpected 

Expected 
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VI.  CONCLUSION 


Our  thesis  asked  whether  social  media  analysis  could  be  a  reliable  proxy  for 
polling  in  a  conflict  zone.  We  decided  to  pursue  this  research  question  after  identifying 
several  gaps.  First,  we  examined  the  importance  of  traditional  collection  and  assessment 
methods  in  detennining  a  population’s  attitudes,  beliefs,  and  opinions,  specifically  in 
low-intensity  conflicts.  During  our  research,  we  identified  a  deficiency  in  the  U.S. 
military’s  doctrinal  assessment  methods.  Primarily,  we  noted  a  need  to  evaluate  popular 
support  more  often  than  traditional  methods  afford,  especially  in  areas  outside  major 
combat  operations. 

This  subsequently  led  us  to  examine  different  ways  social  media  data  is  being 
used,  and  we  identified  sentiment  analysis  as  the  most  appropriate  way  to  evaluate  this 
data.  However,  there  was  one  specific  aspect  missing  from  the  existing  literature:  the 
assessment  of  the  relationship  between  geo-spatially  anchored  social  media  users  and 
traditional  polling  methods.  In  response,  we  designed  an  approach  centered  around 
examining  geo-spatially  anchored  data,  allowing  us  to  assess  whether  people  in  the  same 
general  location  express  the  same  sentiment  in  both  measurement  approaches. 

During  the  analysis  phase  of  our  research,  we  examined  dependent  variables 
derived  from  existing  polling  data,  alongside  newly  developed  independent  variables 
derived  from  Twitter  messages.  What  we  found  in  response  to  our  question  and  research 
method  was  something  unexpected — sentiment  analysis  of  Twitter  data  has  an  additive 
positive  value  when  compared  to  polling  data,  and  metrics  derived  from  Twitter 
messages  may  offer  a  reliable  source  of  data  under  certain  conditions. 
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VII.  ADDITIONAL  RESEARCH 


Our  team  identified  two  broad  areas  of  potential  future  research.  They  are 
sentiment  research  and  operational  changes.  The  following  recommended  research  areas 
are  not  limited  in  value  to  the  U.S.  government. 

A.  IMPACT  OF  SENTIMENT 

Additional  research  is  required  to  find  out  whether  sentiment  or  topic  selection 
under  different  conditions  produces  more  reliable  results  as  noted  in  this  research.  We 
believe  it  would  be  beneficial  to  determine  whether  sentiment  dictionaries  other  than  the 
ones  we  chose  would  provide  different  research  outcomes.  In  particular,  we  recommend 
additional  research  to  detennine  the  impact  of  sentiment  dictionary  lists.  We  recommend 
using  the  same  data  and  approach  to  establish  a  baseline  in  order  to  compare  as  many  lists 
as  are  available.  We  also  recommend  developing  AFINN  dictionaries,  at  least  for  all 
high-interest  areas.  An  Arabic  AFINN  would  have  been  immeasurably  helpful  for  this 
research  and  would  have  eliminated  the  need  for  translation.  We  recognize  the  potential 
negative  impact  of  translation,  so  further  research  is  also  required  to  improve  automated 
translation.  This  is  especially  true  for  the  languages  spoken  in  conflict  areas  and/or  high- 
interest  areas. 

B.  EVALUATION  OF  THE  TOPIC 

Our  research  indicates  that  the  reliability  and  relationship  of  sentiment  results 
vary  by  topic.  However,  only  four  topics  were  evaluated.  What  is  needed  is  a  deeper 
examination  of  which  topics  generate  more  reliable  results  using  this  data  and  data  for 
other  countries.  This  will  help  analysts  identify  gaps  in  social  media  data  reliability. 

C.  DOES  THE  COUNTRY/REGION  MATTER? 

We  believe  it  is  important  to  determine  whether  our  research  method  can  be 
duplicated  in  other  countries  or  regions.  In  particular,  are  there  different  outcomes  for 
countries  that  are  at  peace  versus  those  in  conflict? 
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D.  GEO-LOCATION 

One  explanation  for  the  statistically  insignificant  results  and  inverse  relationships 
found  in  our  research  is  that  there  simply  were  not  enough  tweets  in  Yemen.  Increasing 
the  number  of  available  geo-located  data  points  would  improve  the  reliability  of  our 
research.  Additionally,  more  data  could  help  shed  light  on  the  role  online  obfuscation 
plays.  When  online  users  obscure  their  location  or  identity,  does  this  change  the  content 
of  their  tweets?  Are  they  more  or  less  honest?  Does  this  impact  the  sentiment  and  focus 
of  criticism  or  praise  for  the  govermnent  or  insurgent  groups?  Along  the  same  lines,  the 
USG  should  require  latitude  and  longitude  for  all  future  contracted  and  executed  polls  in 
order  to  conduct  robust  geo-spatial  analyses. 

E.  INTEGRATION  OF  SOCIAL  MEDIA  ANALYSIS 

As  noted  at  the  beginning  of  the  chapter  on  results,  social  media  data  combined 
with  polling  data  produces  a  better  and  more  reliable  model  of  analysis.  Analysts  should 
conduct  social  media  analysis  and  combine  the  results  with  polling  analysis  to  support 
operational  assessments. 
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APPENDIX.  CONCEPT  DICTIONARY 


Extremism  & 
AQAP 

Extremism  & 

AO 

Extremism  & 

AS 

Extremism  & 

AA 

AQAP 

AQ 

Ansar  al  Shariah 

Ansar  Allah 

AQ  in  Yemen 

al-Qaeda 

AAS 

Houthi  militants 

al-Qaeda  in  Yemen 

al  zawahiri 

ASY 

Houthi  fighters 

al-Qaeda  in  the 
Arabian  Peninsula 

Hamza  al  Zinjibari 

Houthi  rebels 

Qasim  al-Raymi 

Jalal  Baleedi 

Houthi  militia 

Sheikh  al-Raymi 

Hamza  al-Zinjibari 

Yahia  Badreddin  al- 
Houthi 

Emir  al-Raymi 

Abdul  Malik 
Badreddin 

Nasir  al-Wuhayshi 

Abu  Ali  Abdullah  al 
Hakem 

Sheikh  al-Wuhayshi 

Emir  al-Wuhayshi 

Extremism  & 
Jihad 

Extremism  & 
Sharia 

Aden-Abyan 

Sharia 

Islamic  Army 

Islamic  Court 

Abdul-Karim 

Jihad 

Wahhabi 

Salafi 

Institution  & 
Military 

Institution  & 
Military  cont’d 

Institution  & 
Police 

Institution  & 
National  govt 

Yemeni  Military 

Yemeni  Air  Force 

Law  enforcement 

ROYG 

Our  Military 

Y emeni  Airmen 

Central  Security 

Government 

Our  Military 
Leadership 

Yemeni  Airman 

Police 

President 

Yemeni  military 
leaders 

Our  Air  Force 

Security  Forces 

Prime  Minister 

Military  Council 

Our  Airmen 

Government  forces 

Parliament 

Military  Leaders 

Yemeni  Fighter 

Jets 

Political  Security 
Organization 

Yemeni  Council 

Yemeni  Security 

Forces 

Our  Fighter  Jets 

National  Security 
Bureau 

Yemeni  Regime 

Yemeni  Armed 

Yemeni  Navy 

Counter-Terrorism 

Republic  of 
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Forces 

Unit 

Yemen 

Our  Security  Forces 

Our  Navy 

Ministry  of  Interior 

Political  Leaders 

Our  Armed  Forces 

Our  Sailors 

Minister  of  Interior 

Our  Troops 

Yemeni 

Battleship 

Criminal 

Investigation 

Yemeni  Troops 

Our  Forces 

Abdel-Qader  Qahtan 

Yemeni  Army 

Our  Government 
Forces 

Abdel  Qader  Qahtan 

Our  Army 

Yemeni  Forces 

CSF 

Our  Soldiers 

Yemeni 

Government 

Forces 

NSB 

Yemeni  Soldiers 

Yemeni 

Republican  Guard 

CTU 

Our  Republican 

Guard 

The  Ministry  of 
Defense  in  Aden 

Yemeni  Ministry  of 
Defense 

Our  Ministry  of 
Defense 

Yemeni  Minister  of 
Defense 

Mohammed 

Nasser  Ahmed 

Institution  & 
Local  govt 

Institution  & 
National  govt 

Institution  & 
Federal  Court 

Institution  & 
Tribal  Justice 

Focal  government 

ROYG 

Justice  System 

Tribal  justice 

City  government 

Government 

Court 

Tribal  court 

Mayor 

President 

Judges 

City  Manager 

Prime  Minister 

City  Council 

Parliament 

My  Municipality 

Yemeni  Council 

Municipal  council 

Yemeni  Regime 

Institution  & 
Bank 

Institution  & 
Elections 

Institution  & 

State  govt 

Yemeni  bank 

Electoral 

Governor 

SBYB 

Election 

Notes:  The  Extremism  annotation  in  the  first  table  indicates  that  all  these  terms  are 
associated  with  the  concept  of  extremism  in  addition  to  the  6  subcategories  of  extremism. 
The  Institution  annotation  in  the  second  table  indicates  that  all  these  terms  are  associated 
with  the  concept  of  institution  in  addition  to  the  10  subcategories  of  Yemeni  institutions. 
This  is  not  an  exhaustive  list  with  spelling  and  punctuation  variations  removed  along  with 
the  Arabic  translation. 
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